test-debug-failures

What this skill does

# Debug Test Failures

## When to Use This Skill

Use this skill when users mention:
- "tests are failing"
- "pytest errors"
- "test suite not passing"
- "debug test failures"
- "fix broken tests"
- "tests failing after changes"
- "mock not called"
- "assertion error in tests"
- Any test-related debugging task

## What This Skill Does

Provides systematic evidence-based test debugging through 6 mandatory phases:

1. **Evidence Collection** - Run tests with verbose output, capture actual errors
2. **Root Cause Analysis** - Categorize problems (test setup vs business logic vs integration)
3. **Specific Issue Location** - Identify exact file:line:column for ALL occurrences
4. **Systematic Fix** - Plan and implement fixes for all related errors
5. **Validation** - Re-run tests and verify no regressions
6. **Quality Gates** - Run project-specific checks (type, lint, dead code)

Prevents assumption-driven fixes by enforcing proper diagnostic sequence.

## Quick Start

When tests fail, use this skill to systematically identify and fix root causes:

```bash
# This skill will guide you through:
# 1. Running tests with verbose output
# 2. Parsing actual error messages
# 3. Identifying root cause (test setup vs business logic vs integration)
# 4. Locating specific issues (file:line:column)
# 5. Fixing ALL occurrences systematically
# 6. Validating the fix
# 7. Running quality gates
```

## Table of Contents

### Core Sections
- [Purpose](#purpose) - Evidence-based systematic debugging philosophy
- [Quick Start](#quick-start) - Immediate workflow overview
- [Mandatory Debugging Sequence](#mandatory-debugging-sequence) - 6-phase systematic approach
  - [Phase 1: Evidence Collection](#phase-1-evidence-collection) - Run tests, capture output, parse errors
  - [Phase 2: Root Cause Analysis](#phase-2-root-cause-analysis) - Categorize problems, check test vs code
  - [Phase 3: Specific Issue Location](#phase-3-specific-issue-location) - Exact locations, find all occurrences
  - [Phase 4: Systematic Fix](#phase-4-systematic-fix) - Plan, implement, address all related errors
  - [Phase 5: Validation](#phase-5-validation) - Re-run tests, check results, full test suite
  - [Phase 6: Quality Gates](#phase-6-quality-gates) - Project checks, verify all gates pass
- [Anti-Patterns](#anti-patterns---stop-if-you-see-these) - Common mistakes to avoid
- [Common Test Failure Patterns](#common-test-failure-patterns) - Diagnostic reference
  - [Pattern 1: Mock Not Called](#pattern-1-mock-not-called) - Execution path issues
  - [Pattern 2: Attribute Error on Mock](#pattern-2-attribute-error-on-mock) - Mock configuration issues
  - [Pattern 3: Assertion Mismatch](#pattern-3-assertion-mismatch) - Business logic errors
  - [Pattern 4: Import/Fixture Errors](#pattern-4-importfixture-errors) - Dependency issues

### Framework-Specific Guides
- [Framework-Specific Quick Reference](#framework-specific-quick-reference) - Test runner commands and flags
  - **Python (pytest)** - Run tests, common flags, debugging options
  - **JavaScript/TypeScript (Jest/Vitest)** - Jest flags, Vitest reporter options
  - **Go** - Test commands, race detection, specific package testing
  - **Rust** - Cargo test, output control, sequential execution

### Advanced Topics
- [Success Criteria](#success-criteria) - Task completion checklist
- [Examples](#examples) - Detailed walkthroughs (see examples.md)
- [Related Documentation](#related-documentation) - Project CLAUDE.md, quality gates, testing strategy
- [Philosophy](#philosophy) - Evidence-based debugging principles

## Instructions

**YOU MUST follow this sequence. No shortcuts.**

### Phase 1: Evidence Collection

**1.1 Run Tests First - See Actual Errors**

DO NOT make any changes before running tests. Execute with maximum verbosity:

**Python/pytest:**
```bash
uv run pytest <failing_test_path> -v --tb=short
# Or for full stack traces:
uv run pytest <failing_test_path> -vv --tb=long
# Or for specific test function:
uv run pytest <file>::<test_function> -vv
```

**JavaScript/TypeScript:**
```bash
# Jest
npm test -- --verbose --no-coverage <test_path>

# Vitest
npm run test -- --reporter=verbose <test_path>

# Mocha
npm test -- --reporter spec <test_path>
```

**Other Languages:**
```bash
# Go
go test -v ./...

# Rust
cargo test -- --nocapture --test-threads=1

# Ruby (RSpec)
bundle exec rspec <spec_path> --format documentation
```

**1.2 Capture Output**

Save the COMPLETE output. Do not summarize. Do not assume.

**1.3 Read The Error - Parse Actual Messages**

Identify:
- Error type (AssertionError, AttributeError, ImportError, etc.)
- Error message (exact wording)
- Stack trace (which functions called which)
- Line numbers (where error originated)

**CRITICAL:** Is this a "mock not called" error or an actual assertion failure?

### Phase 2: Root Cause Analysis

**2.1 Categorize The Problem**

**Test Setup Issues:**
- Mock configuration incorrect (`mock_X not called`, `mock has no attribute Y`)
- Fixture problems (missing, incorrectly scoped, teardown issues)
- Test data issues (invalid inputs, wrong test doubles)
- Import/dependency injection errors

**Business Logic Bugs:**
- Assertion failures on expected values
- Logic errors in implementation
- Missing validation
- Incorrect algorithm

**Integration Issues:**
- Database connection failures
- External service unavailable
- File system access problems
- Environment configuration missing

**2.2 Check Test vs Code**

Use Read tool to examine:
1. The failing test file
2. The code being tested
3. Related fixtures/mocks/setup

**CRITICAL QUESTION:** Is the problem in how the test is written, or what the code does?

### Phase 3: Specific Issue Location

**3.1 Provide Exact Locations**

For EVERY error, document:
- **File:** Absolute path to file
- **Line:** Exact line number
- **Column:** Character position (if available)
- **Function/Method:** Name of failing function
- **Error Type:** Specific exception/assertion
- **Actual vs Expected:** What was expected, what was received

**Example:**
```
File: /abs/path/to/test_service.py
Line: 45
Function: test_process_data
Error: AssertionError: mock_repository.save not called
Expected: save() called once with data={'key': 'value'}
Actual: save() never called
```

**3.2 Identify ALL Occurrences**

Use Grep to find all instances of the pattern:

```bash
# Find all similar test patterns
grep -r "pattern_causing_error" tests/

# Find all places where mocked function is used
grep -r "mock_function_name" tests/
```

**DO NOT fix just the first occurrence. Fix ALL of them.**

### Phase 4: Systematic Fix

**4.1 Plan The Fix**

Before making changes, document:
- What needs to change
- Why this fixes the root cause
- How many files/locations affected
- Whether this is test code or business logic

**4.2 Implement Fix**

**Use MultiEdit for multiple changes to same file:**

```python
# ✅ CORRECT - All edits to same file in one operation
MultiEdit("path/to/file.py", [
    {"old_string": incorrect_mock_setup_1, "new_string": correct_mock_setup_1},
    {"old_string": incorrect_mock_setup_2, "new_string": correct_mock_setup_2},
    {"old_string": incorrect_assertion, "new_string": correct_assertion}
])
```

**For changes across multiple files, use Edit for each file:**

```python
# Fix in test file
Edit("tests/test_service.py", old_string=..., new_string=...)

# Fix in implementation
Edit("src/service.py", old_string=..., new_string=...)
```

**4.3 Address All Related Errors**

If error appears in 10 places, fix all 10. No arbitrary limits.

If pattern appears across files:
1. Use Grep to find all occurrences
2. Document each location
3. Fix each location
4. Track completion

### Phase 5: Validation

**5.1 Re-run Tests**

Run the EXACT same test command from Phase 1:

```bash
# Same command as before
uv run pytest <failing_test_path> -vv
```

**5.2 Check Results**

- ✅ **PASS:** All tests green → Proceed to Phase 6
- ❌ **FAIL:** New errors → Return to Phase 2 (different root caus