test-gap-analysis

What this skill does

# Test Gap Analysis via Pseudo-Mutation

Analyze production code in any supported language by reasoning about hypothetical mutations and checking whether existing tests would catch them. This reveals blind spots where tests pass but would continue to pass even if the code were broken.

> **Language-specific guidance**: Call the `test-analysis-extensions` skill to discover available extension files, then read the file matching the target codebase (e.g., `extensions/dotnet.md`, `extensions/python.md`, `extensions/typescript.md`). The extension file helps you find test files, recognize framework-specific assertion APIs, and identify language-specific null/None/nil patterns and error-handling idioms that map to the mutation catalog below.

## Why Pseudo-Mutation Matters

Code coverage tells you what code ran during tests. It does **not** tell you whether tests would fail if that code were wrong. A method can have 100% line coverage but zero tests that would catch a sign flip, an off-by-one error, or a removed null check.

Pseudo-mutation analysis asks: _"If I changed this line, would any test fail?"_ When the answer is "no," you've found a test gap.

| Coverage Metric | What It Measures | What It Misses |
|----------------|-----------------|----------------|
| Line coverage | Which lines executed | Whether assertions verify those lines' behavior |
| Branch coverage | Which branches taken | Whether both branches produce different asserted outcomes |
| **Mutation score** | Whether tests detect code changes | Nothing — this is the gold standard |

This skill performs **static pseudo-mutation** — reasoning about mutations without actually running them — to approximate mutation testing at the speed of code review.

## When to Use

- User asks "would my tests catch a bug in this code?"
- User wants to find weak or shallow tests
- User wants to evaluate test effectiveness beyond coverage
- User asks for mutation testing or mutation analysis
- User asks "where are my tests blind?"
- User wants to prioritize which tests to strengthen

## When Not to Use

- User wants to write new tests from scratch (use `code-testing-agent` for any language, or `writing-mstest-tests` for MSTest specifically)
- User wants to detect test anti-patterns like flakiness or poor naming (use `test-anti-patterns`)
- User wants to measure assertion variety (use `assertion-quality`)
- User wants to run an actual mutation testing framework (Stryker for .NET/JS/TS, mutmut for Python, PIT for Java, go-mutesting for Go, cargo-mutants for Rust, mutant for Ruby) — help them directly with the tool
- User only wants code coverage numbers (out of scope)

## Inputs

| Input | Required | Description |
|-------|----------|-------------|
| Production code | Yes | The source files to analyze for mutation points |
| Test code | Yes | The test files that cover the production code |
| Focus area | No | A specific mutation category or code region to focus on |

## Workflow

### Step 1: Detect language and load extension

Identify the target codebase's language and test framework. Call the `test-analysis-extensions` skill and read the matching extension file. The mutation catalog below uses language-neutral concepts; the extension file tells you how each concept maps in the language you are analyzing (e.g., `null` vs `None` vs `nil` vs `undefined`, `throw` vs `raise` vs `panic!` vs `return err`).

### Step 2: Gather production and test code

Read both the production code and its corresponding test files. If the user points to a directory, identify production/test pairs by convention — defaults differ by language: `.cs` ↔ `*Tests.cs`/`*.Tests.cs` (.NET), `foo.py` ↔ `test_foo.py`/`foo_test.py` (Python), `foo.ts` ↔ `foo.test.ts`/`foo.spec.ts` (JS/TS), `Foo.java` ↔ `FooTest.java`/`FooTests.java` (Java), `foo.go` ↔ `foo_test.go` (Go), `foo.rb` ↔ `foo_spec.rb`/`test_foo.rb` (Ruby), `lib.rs` ↔ inline `#[cfg(test)] mod tests` or `tests/foo.rs` (Rust), `Foo.swift` ↔ `FooTests.swift` (Swift), `Foo.kt` ↔ `FooTest.kt`/`FooSpec.kt` (Kotlin), `Foo.ps1` ↔ `Foo.Tests.ps1` (Pester), `foo.cpp` ↔ `foo_test.cpp`/`test_foo.cpp` (C++).

Establish which production methods are exercised by which test methods — trace this through method calls in test code, setup, helper methods, and shared examples.

### Step 3: Identify mutation points

Scan the production code and annotate every location where a mutation could reveal a test gap. Use the mutation catalog below.

#### Boundary Mutations

| Original | Mutation | What it tests |
|----------|----------|---------------|
| `<` | `<=` | Off-by-one at upper bound |
| `>` | `>=` | Off-by-one at lower bound |
| `<=` | `<` | Boundary inclusion |
| `>=` | `>` | Boundary inclusion |
| `== 0` | `== 1` or `<= 0` | Zero-boundary handling |
| `i < length` | `i < length - 1` or `i <= length` | Loop boundary |
| `index + 1` | `index` or `index + 2` | Index arithmetic |

#### Boolean and Logic Mutations

| Original | Mutation | What it tests |
|----------|----------|---------------|
| `&&` | `\|\|` | Condition independence |
| `\|\|` | `&&` | Condition necessity |
| `!condition` | `condition` | Negation correctness |
| `if (x)` | `if (!x)` | Branch selection |
| `true` (constant) | `false` | Hardcoded assumption |
| `flag \|\| other` | `other` | Short-circuit first operand |

#### Return Value Mutations

| Original | Mutation | What it tests |
|----------|----------|---------------|
| `return result` | `return null` / `return None` / `return nil` / `return undefined` | Null/None/nil handling downstream |
| `return result` | `return default(T)` / `return T()` / `return ""` / `return 0` | Default value handling |
| `return true` | `return false` | Boolean return verification |
| `return list` | `return new List<T>()` / `return []` / `return Array.Empty<T>()` / `return make([]T, 0)` / `return Vec::new()` / `return @[]` | Empty collection handling |
| `return count` | `return 0` or `return count + 1` | Numeric return verification |
| `return string` | `return ""` or `return null`/`None`/`nil` | String return verification |
| `return Ok(x)` | `return Err(...)` (Rust) | Result/error variant |
| `return value, nil` | `return zero, err` (Go) | Error tuple |

#### Exception / Error Removal Mutations

| Original | Mutation | What it tests |
|----------|----------|---------------|
| `throw new ArgumentNullException(...)` (.NET) / `raise ValueError(...)` (Python) / `throw new Error(...)` (JS) / `throw new IllegalArgumentException(...)` (Java) / `panic!(...)` (Rust) / `panic(...)` (Go) / `raise ArgumentError` (Ruby) / `throw RuntimeException(...)` (Kotlin) / `throw FooError.bar` (Swift) / `throw "..."` (Pester) / `throw std::invalid_argument(...)` (C++) | _(remove entire throw/raise/panic)_ | Guard clause verification |
| `if (x == null) throw ...` / `if x is None: raise ...` / `if (!x) throw ...` / `if x == nil { return err }` (Go) / `assert!(x.is_some())` (Rust) | _(remove entire guard)_ | Null/None/nil guard testing |
| `if (!IsValid()) throw ...` / `if not is_valid(): raise ...` / etc. | _(remove entire check)_ | Validation testing |
| `return err` after error check (Go) | _(remove or swallow error)_ | Error propagation |
| `?` operator (Rust) | `.unwrap()` or `.expect(...)` | Error short-circuit |

#### Arithmetic Mutations

| Original | Mutation | What it tests |
|----------|----------|---------------|
| `a + b` | `a - b` | Addition correctness |
| `a - b` | `a + b` | Subtraction correctness |
| `a * b` | `a / b` | Multiplication correctness |
| `a / b` | `a * b` | Division correctness |
| `a % b` | `a / b` | Modulo correctness |
| `x++` | `x--` | Increment direction |
| `-value` | `value` | Sign flip |

#### Null / None / Nil-Check Removal Mutations

| Original | Mutation | What it tests |
|----------|----------|---------------|
| `if (x == null) return ...` / `if x is None: return ...` / `if (!x) return ...` / `if x == nil { return ... }` / `unless x; return; end` (Ruby) / `if x.is_none() { return ... }`