superval

What this skill does

# Superval - Plan-Driven Validation Loop

> Version: 1.0.0 by skulto

## Overview

Superval is a **plan-driven validation engine** that proves a built project matches its plan. It reads the plan, reads all build state, detects the test framework, and validates at three levels (structural, wiring, behavioral). For behavioral verification, it writes **outside-in black-box acceptance tests** -- independent scripts (often bash or a scripting language) that automate the **built application from the outside**, never importing source code. It loops until everything passes. It never stops trying.

**Core principle:** The plan is the specification. The built code is the implementation. Superval is the proof. Acceptance tests treat the app as a black box -- they poke it from the outside, through its public interface, like a real user would.

**Position in pipeline:**
```
/superplan -> /superbuild or /autobuild -> /superval
  (plan)         (build)                    (validate)
```

---

## When to Use

- After `/superbuild` or `/autobuild` completes all phases
- When you need proof that every planned feature exists and works
- When a build failed partway and you need to assess what's missing
- When resuming after context compaction and need to verify state
- Before creating a PR to prove the implementation is correct

## When NOT to Use

- Before a plan exists (use `/superplan` first)
- During active building (use `/superbuild` or `/autobuild`)
- For projects without a plan document (nothing to validate against)

---

## Execution Flow

```dot
digraph superval {
  rankdir=TB;
  node [shape=box, style=rounded];

  ingest [label="1. INGEST PLAN\nFind and read plan document"];
  state [label="2. READ STATE\nLoad .autobuild/ and plan checkboxes"];
  detect [label="3. DETECT STACK\nFind test framework and tools"];
  no_framework [label="ABORT\nNo test framework found.\nAdvise: /superplan bootstrap\nthe testing pyramid", shape=octagon, style="rounded,filled", fillcolor="#ffcccc"];
  extract [label="4. EXTRACT FEATURES\nBuild feature map from plan"];
  structural [label="5. STRUCTURAL VERIFICATION\nDo expected files exist?"];
  wiring [label="6. WIRING VERIFICATION\nAre modules connected?"];
  behavioral [label="7. BEHAVIORAL VERIFICATION\nDo features actually work?"];
  report [label="8. TRACEABILITY REPORT\nMap every feature to result"];
  all_pass [label="ALL PASS?\nEvery feature verified?", shape=diamond];
  done [label="VALIDATION COMPLETE\nReport: PASS", shape=doubleoctagon, style="rounded,filled", fillcolor="#ccffcc"];
  feedback [label="9. GENERATE FEEDBACK\nStructured failure diagnostics"];
  fix [label="10. FIX FAILURES\nAddress each failure"];
  no_plan [label="ABORT\nNo plan found", shape=octagon, style="rounded,filled", fillcolor="#ffcccc"];

  ingest -> state [label="plan found"];
  ingest -> no_plan [label="no plan"];
  state -> detect;
  detect -> no_framework [label="no test\nframework"];
  detect -> extract [label="framework\ndetected"];
  extract -> structural;
  structural -> wiring;
  wiring -> behavioral;
  behavioral -> report;
  report -> all_pass;
  all_pass -> done [label="yes"];
  all_pass -> feedback [label="no"];
  feedback -> fix;
  fix -> structural [label="re-validate\n(loop forever)"];
}
```

---

## Phase Reference Index

Read the reference doc BEFORE executing that phase:

| Phase | Reference Document | When to Read |
|---|---|---|
| 1. Ingest Plan | `references/PLAN-PARSING.md` | Before parsing any plan |
| 2. Read State | `references/STATE-FILE-CONTRACTS.md` | Before reading .autobuild/ |
| 3. Detect Stack | `scripts/detect-test-framework.sh` | Run this script |
| 4-7. Verification | `references/VALIDATION-PATTERNS.md` | Before any verification |
| 5-7. Test Generation | `references/CLI-TESTING-PATTERNS.md` | Before writing any test |

---

## Phase 1: INGEST PLAN

**Find the plan document.** Search in this order:

1. User-provided path (if given as argument to `/superval`)
2. `docs/*-plan.md` or `docs/*-plan-*.md`
3. Root-level `*-plan.md`
4. `.autobuild/config.json` -> `plan_path` field

**If no plan found:** ABORT immediately.

```
SUPERVAL ABORT: No plan found.

Searched:
  - docs/*-plan.md
  - docs/*-plan-*.md
  - .autobuild/config.json

To create a plan, run: /superplan <feature description>
```

**If plan found:** Read the entire plan. Output confirmation:

```
SUPERVAL: Plan loaded
Plan: docs/autobuild-plan.md
Phases: 6 (0, 1, 2A, 2B, 2C, 3)
Acceptance Criteria: 4
```

**Multi-file plans:** If plan is split across files (`*-plan-1.md`, `*-plan-2.md`), read ALL parts.

---

## Phase 2: READ STATE

Load all available build state to understand what was attempted.

### 2a. Check for .autobuild/ directory

If `.autobuild/` exists (project was built with `/autobuild`):

1. Read `.autobuild/config.json` -> extract stack, commands, phase counts
2. Read each `.autobuild/phases/phase-*.json` -> extract per-phase status, file lists, quality gate results
3. Read `.autobuild/logs/execution.log` -> understand execution timeline

### 2b. Check plan document checkboxes

Read the plan document for superbuild-style state:

1. Phase Overview table -> Status column (⬜/✅/🔄)
2. Per-phase objectives -> `- [x]` vs `- [ ]` counts
3. Per-phase Definition of Done -> `- [x]` vs `- [ ]` counts

### 2c. Output state summary

```
SUPERVAL: State loaded
Source: .autobuild/ + plan checkboxes

Phase Status:
  Phase 0: Bootstrap ......... complete (autobuild verified)
  Phase 1: Core Services ...... complete (autobuild verified)
  Phase 2A: Backend API ....... complete (autobuild verified)
  Phase 2B: Frontend .......... complete (autobuild verified)
  Phase 2C: Tests ............. complete (autobuild verified)
  Phase 3: Integration ........ complete (autobuild verified)

Files expected: 24 created, 8 modified
Quality gates claimed: ALL PASS

NOTE: All claims will be independently verified.
```

---

## Phase 3: DETECT STACK

Run the detection script or perform manual detection.

### Using the script

```bash
./scripts/detect-test-framework.sh <project-dir>
```

### Manual detection (if script unavailable)

Check for these files in order:

| File | Stack |
|---|---|
| `package.json` + `tsconfig.json` | TypeScript |
| `package.json` | JavaScript |
| `pyproject.toml` / `requirements.txt` | Python |
| `go.mod` | Go |
| `Cargo.toml` | Rust |

Then check for test framework:

| Stack | Config Files to Check |
|---|---|
| TypeScript | `vitest.config.ts`, `jest.config.ts`, package.json deps |
| Python | `pytest.ini`, `pyproject.toml` [tool.pytest] |
| Go | Built-in (`go test`) |
| Rust | Built-in (`cargo test`) |

### No test framework found: ABORT

```
SUPERVAL ABORT: No test framework detected.

Stack: typescript
Checked: vitest.config.ts, jest.config.ts, package.json

Cannot validate without a test framework.
To bootstrap testing, run: /superplan bootstrap the testing pyramid for me
```

**This is a hard stop. Do NOT proceed without a test framework.**

### Framework found: Continue

```
SUPERVAL: Stack detected
Stack: typescript
Package Manager: npm
Test Framework: vitest
Linter: eslint
Formatter: prettier
Type Checker: tsc
Test Command: npm test
Test Files Found: 12
```

---

## Phase 4: EXTRACT FEATURES

Parse the plan to build the complete feature map. See `references/PLAN-PARSING.md` for parsing details.

### Extract from plan:

1. **Phase Overview table** -> all phases with names and status
2. **Per-phase Objectives** -> feature checklist per phase
3. **Per-phase Code Changes** -> expected files (CREATE/MODIFY/DELETE)
4. **Per-phase Tests** -> expected test files
5. **Acceptance Criteria** -> high-level feature requirements
6. **Definition of Done** -> quality gate requirements per phase

### Build the feature map:

For each phase, create a feature entry:

```
Feature: Phase 1 - Core Services
  Objectives: [config service, logger service, state service]
  Files Created: [src/services/config.ts, src/services/logger.ts, src/services/state.ts]
  Files Mo