checklist-runner

What this skill does

# Checklist Runner

Parse and verify GitHub PR/issue checklists, auto-checking items that pass verification. Classifies each checklist item, runs the cheapest verification possible, and checks off what passes — asking the user only when truly necessary.

## Core Principles

1. **Classify Before Executing** — Categorize every item before verification to pick the cheapest strategy
2. **CI-First for Tests** — Check CI status before running locally; avoid redundant work
3. **Confidence-Based Automation** — HIGH confidence items auto-proceed; MEDIUM/LOW confidence items pause for user
4. **Ownership-Aware Updates** — Respect GitHub permissions; never silently modify someone else's content
5. **Safe Operations** — Use `updated_at` to prevent race conditions; default to comment mode for others' posts

## Quick Start

```text
/checklist             # Auto-detect current branch's PR
/checklist #123        # Specific PR or issue number
/checklist <url>       # Specific PR or issue URL
```

## When to Use

- User asks to "verify the checklist", "check off items", "run the checklist", "process checklist"
- A PR or issue contains `- [ ]` unchecked items that need verification
- User wants to auto-check items that pass verification before merging
- User wants a verification report on checklist completion status

**When NOT to use**: For creating new checklists from scratch (just write markdown). For PR review comments, use `pr-comment-resolver` instead.

## Workflow Overview

| Phase | Name | Purpose |
|-------|------|---------|
| 1 | Source Resolution + Permission Probe | Auto-detect PR/issue, fetch body + comments, parse checklist items, detect ownership + permissions |
| 2 | Item Classification | Classify each unchecked item into Auto / CI / Shell / Scan / Human |
| 3 | Verification Execution | Execute verifications: Auto (instant) → CI (one-time check) → Shell (grep/jq) → Scan (subagents) → Human (batched questions) |
| 4 | Checkbox Update | Apply ownership rules: own post → auto-check; other's post → suggest or comment mode |
| 5 | Summary Report | Per-item status table, statistics, failed items with evidence, next steps |

### Phase 1: Source Resolution + Permission Probe

Determine the target PR/issue, fetch all checklist sources, and probe permissions upfront.

**Source Resolution Decision Tree:**

```text
Input received
      │
      ├── Explicit URL
      │   └── Parse owner/repo/type/number from URL
      │
      ├── #N (number)
      │   └── gh api repos/{o}/{r}/issues/{n} → detect PR vs issue
      │       (PRs are also issues in GitHub API; check for pull_request key)
      │
      └── Nothing provided
          └── gh pr view --json number,title,body,url
              ├── Success → use current branch's PR
              └── Failure → ask user to specify
```

**Fetch Checklist Sources:**

Collect all checklist items from the PR/issue body and comments. Use `gh pr view` for the PR body and GraphQL for comments. See [verification-recipes.md](references/verification-recipes.md) for exact API endpoints and queries.

> **Pagination**: Paginate when `pageInfo.hasNextPage` is true. See [verification-recipes.md](references/verification-recipes.md) for pagination handling.

> **API field naming**: `gh pr view --json` uses camelCase (`updatedAt`), while REST `gh api` returns snake_case (`updated_at`). Normalize to `updated_at` internally.

**Parse Checklist Items:**

Extract all `- [ ]` and `- [x]` items from each source.

**Important**: Before extracting, strip fenced code blocks (`` ``` ... ``` ``) and inline code spans to avoid parsing example checklists inside code as real items.

Track for each item:
- Item text (normalized, trimmed)
- Checked state (unchecked = needs verification)
- Source (body vs comment ID)
- Source author
- Source `updated_at` timestamp (for race condition prevention in Phase 4)
- Nesting level (indented items are nested under a parent)

**Permission Probe:**

Get current user (`gh api user`), check repo write access (`gh api repos/{o}/{r}`), and compare with each source author. See [checkbox-update-rules.md](references/checkbox-update-rules.md) for full ownership detection, bot detection rules, and permission commands.

> **Null permissions**: If `.permissions` is null or absent (e.g., fine-grained PAT without `metadata:read`), treat as no write access.

**Output update mode to user** (see [checkbox-update-rules.md](references/checkbox-update-rules.md) for the full permission matrix):

```text
Source Analysis:
- PR #123 body: 8 unchecked items, author: @you → auto-check mode
- Comment by @reviewer: 3 unchecked items → suggest-then-check mode
- Total: 11 unchecked items to verify
```

**Edge Cases:**
- PR is closed/merged → warn user; ask if proceed anyway
- No checklist found → report "No checklist items found" and exit
- All items already checked → report; offer to re-verify if user explicitly requests (re-classify and re-verify all items regardless of checked state)
- Nested checklists → flatten with parent context preserved; verify each item independently but note its parent condition

### Phase 2: Item Classification

Classify each unchecked item to determine verification strategy. See [classification-patterns.md](references/classification-patterns.md) for full pattern reference.

**5 Classification Categories:**

| Category | Description | Example Items |
|----------|-------------|---------------|
| **Auto** | Verifiable with file/field checks | "Plugin name has `vp-` prefix", "SKILL.md has valid frontmatter" |
| **CI** | Verifiable via CI status check | "Tests pass", "Lint passes", "Build succeeds" |
| **Shell** | Verifiable with a single grep/find/jq | "No console.log left", "No TODO comments", "No `debugger` statements" |
| **Scan** | Needs semantic understanding (subagent) | "No secrets in code", "Documentation updated", "Changelog entry added" |
| **Human** | Cannot be automatically verified | "Design reviewed", "PM approved", "UX looks good" |

**Classification**: Normalize item text → match against patterns in priority order (Auto > CI > Shell > Scan > Human) → assign confidence. See [classification-patterns.md](references/classification-patterns.md) for the full algorithm, regex patterns, and disambiguation rules.

- HIGH confidence → auto-proceed
- MEDIUM/LOW → present classification to user for confirmation

**Output:**

```text
Item Classification (confidence = how sure we are about the category, not the verification result):
┌───┬─────────────────────────────────────┬──────────┬────────────┐
│ # │ Item                                │ Category │ Confidence │
├───┼─────────────────────────────────────┼──────────┼────────────┤
│ 1 │ Plugin name has `vp-` prefix        │ Auto     │ HIGH       │
│ 2 │ Tests pass                          │ CI       │ HIGH       │
│ 3 │ No console.log statements           │ Shell    │ HIGH       │
│ 4 │ No secrets in code                  │ Scan     │ HIGH       │
│ 5 │ Design reviewed by team             │ Human    │ HIGH       │
│ 6 │ Code quality is good                │ Human    │ LOW        │
└───┴─────────────────────────────────────┴──────────┴────────────┘

⚠️ Item #6 has LOW confidence. Confirm category or reclassify? [Human/Shell/Scan]
```

### Phase 3: Verification Execution

Execute verifications in cost order: Auto (instant) → CI (one API call) → Shell (single command) → Scan (subagents, confirm first) → Human (batched questions).

See [verification-recipes.md](references/verification-recipes.md) for specific commands and subagent prompts.

**Auto Verification:**

Run specific file/field checks. Each produces a definitive PASS/FAIL. See [verification-recipes.md](references/verification-recipes.md) for common recipes and custom recipe construction.

**CI Verification (one-time check, NO polling):**

```bash
gh pr checks <N> --json name,state,bucket
```

| CI State | Action |
|----------|--------|
| All passed | PASS |
| Any failed | FAIL (show which checks failed) |
| Pendin