ln-522-manual-tester

What this skill does

> **Paths:** File paths (`references/`, `../ln-*`) are relative to this skill directory.

**MANDATORY READ:** Load `references/ci_tool_detection.md` — compact output flags, pipefail, and failure-artifact policy for bash/curl/Puppeteer scripts.

## Inputs

| Input | Required | Source | Description |
|-------|----------|--------|-------------|
| `storyId` | Yes | args, git branch, kanban, user | Story to process |

**Resolution:** Story Resolution Chain.
**Status filter:** To Review

# Manual Tester

**Type:** L3 Worker

Manually verifies Story AC on running code and reports structured results for the quality gate.

## Purpose & Scope
- Create executable test scripts in `tests/manual/` folder of target project.
- Run AC-driven checks via bash/curl (API) or puppeteer (UI).
- Save scripts permanently for regression testing (not temp files).
- Document results via the configured tracker provider (`addComment`) with pass/fail per AC and script path.
- No status changes or task creation.

## When to Use
- Use when a Story needs hands-on acceptance-criteria verification before automated planning
- Research comment "## Test Research:" exists on Story (from ln-521)
- All implementation tasks in Story status = Done

## Test Design Principles

### 1. Fail-Fast - No Silent Failures

**CRITICAL:** Tests MUST return 1 (fail) immediately when any criterion is not met.

**Never use:** `print_status "WARN" + return 0` for validation failures, graceful degradation without explicit flags, silent fallbacks that hide errors.

**Exceptions (WARN is OK):** Informational warnings that don't affect correctness, optional features (with clear justification in comments), infrastructure issues (e.g., missing Nginx in dev environment).

### 2. Expected-Based Testing - The Golden Standard

**CRITICAL:** Tests MUST compare actual results against **expected reference files**, not apply heuristics or algorithmic checks.

**Directory structure:**
```
tests/manual/NN-feature/
├── samples/               # Input files
├── expected/              # Expected output files (REQUIRED!)
│   └── {base_name}_{source_lang}-{target_lang}.{ext}
└── test-*.sh
```

**Heuristics acceptable ONLY for:** dynamic/non-deterministic data (timestamps, UUIDs, tokens - normalize before comparison; JSON with unordered keys - use `jq --sort-keys`).

### 3. Results Storage

Test results saved to `tests/manual/results/` (persistent, in .gitignore). Named: `result_{ac_name}.{ext}` or `response_{ac_name}.json`. Inspectable after test completion for debugging.

### 4. Expected File Generation

To create expected files:
1. Run test with current implementation
2. Review output in `results/` folder
3. If correct: copy to `expected/` folder with proper naming
4. If incorrect: fix implementation first, then copy

**IMPORTANT:** Never blindly copy results to expected. Always validate correctness first.

## Workflow

### Phase 0: Resolve Inputs

**MANDATORY READ:** Load `references/input_resolution_pattern.md`

1. **Resolve storyId:** Run Story Resolution Chain per guide (status filter: [To Review]).

### Phase 1: Setup tests/manual structure
1) **Read `docs/project/infrastructure.md`** — get port allocation, service endpoints, base URLs. **Read `docs/project/runbook.md`** — get Docker commands, test prerequisites, environment setup
2) Check if `tests/manual/` folder exists in project root
3) If missing, create structure:
   - `tests/manual/config.sh` — shared configuration (BASE_URL, helpers, colors)
   - `tests/manual/README.md` — folder documentation (see README.md template below)
   - `tests/manual/test-all.sh` — master script to run all test suites (see test-all.sh template below)
   - `tests/manual/results/` — folder for test outputs (add to `.gitignore`)
4) Add `tests/manual/results/` to project `.gitignore` if not present
5) If exists, read existing `config.sh` to reuse settings (BASE_URL, tokens)

### Phase 2: Create Story test script
1) Fetch Story, parse AC into Given/When/Then list (3-5 expected)
   - **Check for research comment** (from ln-521-test-researcher) — incorporate findings into test cases
2) Detect API vs UI (API → curl, UI → puppeteer). **IF UI:** **MANDATORY READ:** Load `references/puppeteer_patterns.md`
3) Create test folder structure:
   - `tests/manual/{NN}-{story-slug}/samples/` — input files (if needed)
   - `tests/manual/{NN}-{story-slug}/expected/` — expected output files (REQUIRED for deterministic tests)
4) Generate test script: `tests/manual/{NN}-{story-slug}/test-{story-slug}.sh`
   - Use appropriate template: TEMPLATE-api-endpoint.sh (direct calls) or TEMPLATE-document-format.sh (async jobs)
   - Header: Story ID, AC list, prerequisites
   - Test function per AC + edge/error cases
   - **diff-based validation** against expected files (PRIMARY)
   - Results saved to `tests/manual/results/`
   - Summary table with timing
5) Make script executable (`chmod +x`)

### Phase 3: Update Documentation
1) Update `tests/manual/README.md`:
   - Add new test to "Available Test Suites" table
   - Include Story ID, AC covered, run command
2) Update `tests/manual/test-all.sh`:
   - Add call to new script in SUITES array
   - Maintain execution order (00-setup first, then numbered suites)

### Phase 4: Execute and report

**MANDATORY READ:** Load `references/test_result_format_v1.md`

1) Rebuild Docker containers (no cache), ensure healthy
2) Run generated script, capture output
3) Parse results (pass/fail counts)
4) Post tracker comment (`addComment`, per test_result_format_v1.md) with:
   - AC matrix (pass/fail per AC)
   - Script path: `tests/manual/{NN}-{story-slug}/test-{story-slug}.sh`
   - Rerun command: `cd tests/manual && ./{NN}-{story-slug}/test-{story-slug}.sh`

## Critical Rules
- Scripts saved to project `tests/manual/`, NOT temp files.
- Rebuild Docker before testing; fail if rebuild/run unhealthy.
- Keep language of Story (EN/RU) in script comments and tracker comment.
- No fixes or status changes; only evidence and verdict.
- Script must be idempotent (can rerun anytime).

## Runtime Summary Artifact

**MANDATORY READ:** Load `references/test_planning_summary_contract.md`, `references/test_planning_worker_runtime_contract.md`

Runtime profile:
- family: `test-planning-worker`
- worker: `ln-522`
- summary kind: `test-planning-worker`
- payload fields used by coordinators: `worker`, `status`, `warnings`, `manual_result_path`

Invocation rules:
- standalone: omit `runId` and `summaryArtifactPath`
- managed: pass both `runId` and exact `summaryArtifactPath`
- always write the validated summary before terminal outcome

Test scripts always go to `tests/manual/`, never to the project root.

### Monitor Integration (Claude Code 2.1.98+)

**MANDATORY READ:** Load `references/monitor_integration_pattern.md`

When running test scripts expected to take >30 seconds:
`Monitor(command="bash tests/manual/{suite}/test-{slug}.sh 2>&1", timeout_ms=300000, description="manual test: {slug}")`

Fallback: if Monitor is unavailable (Bedrock/Vertex), use `Bash(run_in_background=true)`.


## Definition of Done
- [ ] `tests/manual/` structure exists (config.sh, README.md, test-all.sh, results/ created if missing).
- [ ] `tests/manual/results/` added to project `.gitignore`.
- [ ] Test script created at `tests/manual/{NN}-{story-slug}/test-{story-slug}.sh`.
- [ ] `expected/` folder created with at least 1 expected file per deterministic AC.
- [ ] Script uses **diff-based validation** against expected files (not heuristics).
- [ ] Script saves results to `tests/manual/results/` for debugging.
- [ ] Script is executable and idempotent.
- [ ] **README.md updated** with new test suite in "Available Test Suites" table.
- [ ] **test-all.sh updated** with call to new script in SUITES array.
- [ ] App rebuilt and running; tests executed.
- [ ] Verdict and tracker comment posted with script path and rerun command.

## Script Templates

### README.md (created once per project)

```markdown
# Manual Testing Scripts

> **SCOPE