test-smell-detection

What this skill does

# Test Smell Detection

Deep formal audit of test code in any supported language using an academic test smell taxonomy. Detects symptoms of bad design or implementation decisions that make tests harder to understand, more fragile, less effective at catching bugs, or more expensive to maintain. Produces a severity-ranked report with specific locations and actionable fixes.

> **Language-specific guidance**: Call the `test-analysis-extensions` skill to discover available extension files, then read the file matching the target codebase. The extension file documents test markers, sleep / time / random APIs, skip annotations, setup/teardown, mystery-guest indicators (file/database/network/env), integration markers, and language-specific calibration notes that drive the smell detectors below.

## Why Test Smells Matter

Test smells erode confidence in a test suite and inflate maintenance costs:

| Problem | Consequence |
|---------|-------------|
| Tests with conditional logic | Some paths never execute — hidden testing gaps |
| Tests that depend on external resources | Flaky failures, slow execution, environment coupling |
| Tests that sleep to wait for results | Non-deterministic timing, slow suites, false failures |
| Tests without assertions | False confidence — coverage looks good but nothing is verified |
| Tests that call many production methods | Hard to diagnose failures, unclear what's being tested |
| Tests with magic numbers | Unreadable intent, unclear boundary conditions |
| Tests relying on ToString for comparison | Brittle to formatting changes, obscure failure messages |
| Tests with exception handling logic | Swallowed failures, tests that pass when they shouldn't |

## When to Use

- User asks for a comprehensive or formal test smell audit
- User asks "are my tests well-written?" and wants a thorough analysis
- User wants a test quality health check with academic rigor
- User asks for a review of test design or structure using standard smell categories
- User suspects tests are fragile, flaky, or giving false confidence and wants a deep investigation

## When Not to Use

- User wants a quick pragmatic test review (use `test-anti-patterns` — faster, covers the most common issues)
- User wants to evaluate assertion diversity specifically (use `assertion-quality`)
- User wants to find duplicated boilerplate across tests (use `exp-test-maintainability`)
- User wants to write new tests from scratch (help them directly)
- User wants to fix a specific failing test (diagnose and fix directly)

## Inputs

| Input | Required | Description |
|-------|----------|-------------|
| Test code | Yes | One or more test files or a test project directory to analyze |
| Production code | No | The code under test, for context on whether patterns are justified |

## Workflow

### Step 1: Detect language and load extension

Identify the target codebase's language and test framework. Call the `test-analysis-extensions` skill and read the matching extension file (e.g., `extensions/dotnet.md`, `extensions/python.md`, `extensions/typescript.md`, `extensions/go.md`). The extension file lists the framework-specific test markers, sleep / wait APIs, skip / ignore attributes, mystery-guest indicators, and integration-test markers that the smell detectors below need.

### Step 2: Gather the test code

Read all test files the user provides. If the user points to a directory or project, scan for all test files using the markers in the loaded language extension file.

For a thorough audit, also consult the [extended smell catalog](references/test-smell-catalog.md) which covers 9 additional smell types beyond the core 10 below.

### Step 3: Scan for test smells

For each test method and class, check for the following smell categories. Examples reference .NET attributes but the patterns apply across all supported languages — use the loaded language extension file to map each pattern to the framework you are auditing.

#### Smell 1: Conditional Test Logic

Test methods containing `if`, `else`, `switch`, ternary (`? :`), `for`, `foreach`, `while`, or pattern-match arms that change assertion behavior. Control flow in tests means some paths may never execute, hiding gaps.

**Severity:** High
**Detection:** Any control-flow statement inside a test method body that affects which assertions run.
**Exceptions (per-language idioms, do NOT flag):**
- **Foreach-assert** used solely to assert every item in a known collection (the assertion *is* the loop body).
- **Go / Rust table-driven tests**: `for _, tt := range tests { t.Run(tt.name, func(t *testing.T) { ... }) }` (Go) or `#[rstest]` parametrized loops are idiomatic.
- **`it.each(...)` / `test.each(...)` / `@pytest.mark.parametrize` / `[Theory] + [InlineData]` / `@ParameterizedTest`** parametrization driven by data tables.
- **Pester `-ForEach` / `-TestCases`** and **RSpec `where` blocks**.
- **Catch2 `SECTION`s and `GENERATE(...)`**, **doctest `SUBCASE`**, **GoogleTest `INSTANTIATE_TEST_SUITE_P`**.

#### Smell 2: Mystery Guest

Tests that depend on external resources — files on disk, databases, network endpoints, environment variables — without making the dependency explicit or using test doubles.

**Severity:** High
**Detection:** Test methods that read files, open database connections, make HTTP requests (without a test handler), read environment variables, or use hard-coded file paths. Per language: `File.ReadAllText` / `Directory.GetFiles` / `HttpClient` / `Environment.GetEnvironmentVariable` (.NET); `open()` / `pathlib.Path.read_text()` / `requests.get()` / `os.environ[...]` (Python); `fs.readFileSync` / `fetch(...)` / `process.env.X` (JS/TS); `Files.readAllBytes` / `Files.newInputStream` / `HttpClient.send` / `System.getenv` (Java); `os.ReadFile` / `http.Get` / `os.Getenv` (Go); `File.read` / `Net::HTTP.get` / `ENV[...]` (Ruby); `std::fs::read_to_string` / `reqwest::get` / `std::env::var` (Rust); `String(contentsOfFile:)` / `URLSession.shared.data` / `ProcessInfo.processInfo.environment` (Swift); `File(...).readText()` / `URL(...).openConnection()` / `System.getenv` (Kotlin); `Get-Content` / `Invoke-WebRequest` / `$env:X` (Pester); `std::ifstream` / `curl_easy_perform` / `std::getenv` (C++).
**Exception:** In-memory fakes, test-specific handlers, or hermetic test data factories are fine.

#### Smell 3: Sleepy Test

Tests that call sleep or delay functions to wait for a condition. These introduce non-deterministic timing and slow down the suite.

**Severity:** High
**Detection:** Calls to sleep/delay functions inside test methods: `Thread.Sleep` / `Task.Delay` (.NET); `time.sleep` / `asyncio.sleep` (Python); `setTimeout` / `await new Promise(r => setTimeout(...))` / `jest.advanceTimersByTime` not paired with a wait (JS/TS); `Thread.sleep` / `TimeUnit.SECONDS.sleep` (Java); `time.Sleep` (Go); `sleep` / `Kernel#sleep` (Ruby); `std::thread::sleep` / `tokio::time::sleep` (Rust); `Thread.sleep` / `delay` (Kotlin coroutines); `sleep(_:)` / `Task.sleep` (Swift); `Start-Sleep` (Pester); `std::this_thread::sleep_for` (C++). See the matching language extension file for the full list.

#### Smell 4: Assertion-Free Test (Unknown Test)

Tests that execute code but never assert anything. Test frameworks report these as passing even if the code is completely broken, as long as no exception is thrown.

**Severity:** High
**Detection:** A test method with no assertion calls and no expected-exception annotation. Framework-specific: missing `Assert.*` (.NET); no `assert` / `pytest.raises` (Python); no `expect(...)` or `assert.*` (JS/TS); no `assert*` / `assertThat` (Java); no `t.Error*` / `t.Fatal*` / `assert.*` testify (Go); no `expect`/`.to`/`.eq` (RSpec) or `assert*`/`refute*` (Minitest); no `assert*!` / `assert_eq!` / `panic!` (Rust); no `XCTAssert*` / `#expect` (Swift); no `assert*` / `should*` / Kotest matchers (Kotlin); no `Should -*` (Pester); no `EXPECT_*` / `ASSERT_*` / `REQUIRE` / `CHECK` (C++).
**Calibration:**
- A method named `*_