gen-test-plan

What this skill does

# Generate Test Plan

Analyze the repository's tech stack, branch changes vs default, and generate an executable YAML test plan focused on user-facing impact.

**This is an E2E test plan — not an automated test wrapper.** The generated plan will be executed by an autonomous agent acting exactly as a human QA tester would: launching real binaries, hitting real endpoints, interacting with real databases, and verifying real observable behavior.

## Critical Rule: No Automated Test Duplication

**NEVER generate test steps that re-run the project's existing automated test suite.** This means:
- No `cargo test`, `pytest`, `npm test`, `go test`, `mix test`, or equivalent commands as test steps
- No wrapping unit/integration test modules in a test case
- No "run the tests and check they pass" — that's CI's job, not QA's

If you find yourself writing a test step that invokes the project's test runner, **stop and rethink**. Ask: "What would a human tester do to verify this feature works?" The answer is never "run the unit tests."

**What E2E test steps look like:**
- Build the binary and run it with real arguments, check stdout/stderr/exit code
- Start a server and hit it with curl
- Run a CLI command that writes to a real database, then query the database to verify
- Launch the TUI and verify it renders (via screenshot or process lifecycle)
- Chain multiple commands that exercise a full user workflow end-to-end

## Hard gates

Complete these **in order**. Do not advance to the next gate until its **Pass** condition is met (each pass should leave retrievable evidence: pasted command output, a written list, or the generated file on disk). **Scheduling:** Gate **1** before Step **2**; Gate **2** before Step **5**; Gate **3** before Step **7**; Gates **4–5** during **Step 8** (after the Step 7 summary).

1. **Diff and base pinned (after Step 1)** — Resolve the base branch from `--base` when provided, otherwise use the repo default (`main` or `master` per Step 1). Compare `HEAD` to `$(git merge-base HEAD origin/<base_branch>)` (or equivalent if the remote ref differs). **Pass:** You record `current_branch`, `base_branch`, the merge-base SHA or range used, and `changed_files` from `git diff --name-only <merge-base>..HEAD` (empty list allowed if you paste or quote that output and state “no file changes vs base”).

2. **Trace complete (after Step 4)** — **Pass:** Every affected entry point you will test has a **Core functionality** vs **Configuration/admin** classification, and the Step 4 requirement holds: at least one test targets a core entry point **or** you document why that is impossible and flag manual review.

3. **Plan file valid (after Step 6, before Step 7)** — **Pass:** `docs/testing/test-plan.yaml` exists and the following command exits 0 (parses the YAML **and** asserts all four top-level keys are present — a single `grep -E` with alternations would pass on any one match, so do not substitute it):

    ```bash
    python3 -c "import sys, yaml; d = yaml.safe_load(open('docs/testing/test-plan.yaml')) or {}; missing = [k for k in ('version', 'metadata', 'setup', 'tests') if k not in d]; sys.exit('Missing keys: ' + ', '.join(missing) if missing else 0)"
    ```

4. **No automated-test duplication (Step 8)** — **Pass:** Every `run:` step and every `services:` `command:` is scanned for project test runners (`cargo test`, `pytest`, `npm test`, `go test`, `mix test`, `jest`, `vitest`, `mocha`, etc.); **zero** invocations. If any appear, remove or replace them with real E2E actions and re-run Gate 3.

5. **Behavioral coverage (Step 8)** — **Pass:** Re-read `metadata.changes_summary` and recent commit messages; at least one test’s `context`/`steps` exercises the primary user-visible behavior they describe. If they describe a capability (e.g., a new provider) but no step invokes it, add that test or fail verification.

## Arguments

- `--base <branch>`: Base branch to diff against (default: `main`)
- Path: Target directory (default: current working directory)

## Step 1: Gather Repository Context

```bash
# Get current branch
git rev-parse --abbrev-ref HEAD

# Resolve base branch: use --base if supplied, otherwise default (main → master)
BASE_BRANCH="${BASE_BRANCH:-$(git rev-parse --verify origin/main >/dev/null 2>&1 && echo main || echo master)}"
MERGE_BASE="$(git merge-base HEAD "origin/${BASE_BRANCH}")"

# Get changed files vs base
git diff --name-only "${MERGE_BASE}"..HEAD

# Get commit messages for context
git log --oneline "${MERGE_BASE}"..HEAD
```

**Capture:**
- `current_branch`: Branch name
- `base_branch`: Default branch to compare against
- `changed_files`: List of modified files
- `commit_messages`: What the PR is about

## Step 2: Detect Tech Stack

See [references/stack-discovery.md](references/stack-discovery.md) for stack detection commands, entrypoint discovery, port discovery, and trace rules.

## Step 3: Discover User-Facing Entry Points

A "user-facing entry point" is anything a human interacts with: CLI subcommands, HTTP endpoints, UI routes, TUI screens, gRPC services, database migrations, or configuration files that affect runtime behavior.

### CLI Applications (Rust/clap, Python/argparse/click, Go/cobra)

```bash
# Rust (clap) — look for Subcommand derives and command enums
grep -rn "Subcommand\|#\[command\]" --include="*.rs" | head -20

# Python (click/typer/argparse)
grep -rn "@click.command\|@app.command\|add_parser\|add_subparser" --include="*.py" | head -20

# Go (cobra)
grep -rn "cobra.Command\|AddCommand" --include="*.go" | head -20
```

Build a map of:
- CLI subcommands: command name + description + file:line
- Required arguments and flags per subcommand
- Environment variables the binary reads (grep for `env`, `std::env::var`, `os.Getenv`, `os.environ`)

### HTTP/API Services

**Python (FastAPI/Flask):**
```bash
grep -rn "@app\.\(get\|post\|put\|delete\|patch\)" --include="*.py" | head -20
grep -rn "@router\.\(get\|post\|put\|delete\|patch\)" --include="*.py" | head -20
```

**Node.js (Express/Fastify):**
```bash
grep -rn "app\.\(get\|post\|put\|delete\)" --include="*.ts" --include="*.js" | head -20
grep -rn "router\.\(get\|post\|put\|delete\)" --include="*.ts" --include="*.js" | head -20
```

**Rust (axum/actix/rocket):**
```bash
grep -rn "Router::new\|\.route(\|#\[get\]\|#\[post\]\|HttpServer" --include="*.rs" | head -20
```

**Go (net/http, gin, chi):**
```bash
grep -rn "http.HandleFunc\|r.GET\|r.POST\|router.Get\|router.Post" --include="*.go" | head -20
```

**Elixir (Phoenix):**
```bash
grep -rn "get \"/\|post \"/\|pipe_through\|live \"/\|scope \"/\"" --include="*.ex" | head -20
```

### Browser UI Routes

```bash
grep -rn "createBrowserRouter\|<Route\|path=" --include="*.tsx" --include="*.jsx" | head -20
```

### Database and Migrations

```bash
# SQL migrations
ls migrations/ db/migrate/ priv/repo/migrations/ 2>/dev/null
# Schema files
ls schema.sql schema.prisma 2>/dev/null
```

### Build a consolidated map of:
- CLI subcommands: name + args + file:line
- API endpoints: method + path + file:line
- UI routes: path + component + file:line
- Database migrations: filename + what they create/alter
- Configuration: env vars and config files that affect behavior

## Step 4: Trace Changes to Entry Points

For each changed file, determine if it affects user-facing functionality:

1. **Direct entry point change** — File contains route definitions
2. **Import chain analysis** — Find what imports the changed file and trace up to entry points
3. **Architecture-aware tracing** — Read the project's CLAUDE.md, README, or architecture docs to understand data flow and module relationships, rather than relying solely on grep
4. **Document the trace path** in test context

### Import Chain Analysis by Ecosystem

```bash
# Rust — use/mod/crate references and workspace deps
grep -rn "use.*<crate>\|mod <module>" --include="*.rs"
grep -rn "<crate-name>" --include="Cargo.toml"

# Python — from/import
grep -rn "from.*<module>\|import.*<module>"