schema-author

What this skill does

# schema-author — evolve your schema pack

## Non-goals (use these other skills instead)

This skill AUTHORS the schema pack (adds page types, link verbs, prefixes,
flags). For these adjacent jobs, route elsewhere:

- **Filing one specific page** → `skills/brain-taxonomist/SKILL.md`. Brain-
  taxonomist routes at WRITE TIME ("where does this note go?"). schema-author
  changes the rules at AUTHORING TIME ("what types and prefixes exist?").
- **Schema-check as part of EIIRP iteration** → `skills/eiirp/SKILL.md`
  already has a schema-check phase. Don't duplicate.
- **Just looking up a type's settings** → `gbrain schema explain <type>`
  directly. This skill is for CHANGING the pack, not READING from it.
- **Querying who knows about X** → `skills/expert-routing/SKILL.md` (or
  `gbrain whoknows` directly). schema-author makes a type expert-routable;
  it does not run the query.

## Convention

> **Convention:** see [conventions/brain-first.md](../conventions/brain-first.md) for the lookup chain (search → query → get_page → external).

> **Convention:** see [conventions/schema-evolution.md](../conventions/schema-evolution.md) for "when to add a type vs alias vs prefix" — the heuristic.

## When to invoke

Invoke when the user (or a sibling skill) says any of:
- "Add a `researcher` type to my schema"
- "I have 4000 untyped pages under `meetings/`"
- "My brain doesn't know that `journal-article` is a type"
- "Set `paper` to be extractable"
- "Propose types from what I've ingested"
- "Sync the new types to backfill existing pages"

DON'T invoke for "where does THIS note go" (use brain-taxonomist) or
"who knows about X" (use expert-routing / `gbrain whoknows`).

## Tutorial + vision

- **Why this matters:** [`docs/what-schemas-unlock.md`](../../docs/what-schemas-unlock.md) — 7 killer use cases (4000 invisible meetings made queryable, founder ops brain, research brain, legal brain, team brain, agent-as-co-curator) plus the structural argument for why types matter at query time. Read this before pitching schema authoring to a user — it's the doc that explains the difference between a pile of notes and a brain with structure.
- **5-minute walkthrough:** [`docs/schema-author-tutorial.md`](../../docs/schema-author-tutorial.md) — fork the bundled pack, add a researcher type, sync, prove the T1.5 wiring via `gbrain whoknows`. Use placeholder pages so it runs against any brain without affecting real content.

## Workflow

### Phase 1 — Brain (know which pack is active)

```
gbrain schema active --json
```

Output gives you `pack_name`, `version`, `sha8`, `page_types_count`, `source_tier`.
If `source_tier === "default"`, the user is on bundled `gbrain-base` and any
mutation will need a fork first (Phase 4).

### Phase 2 — Assess (what does the current pack cover?)

```
gbrain schema stats --json
```

Returns per-type page counts, untyped count, and `dead_prefixes` (pack-
declared prefixes with zero matching pages — probable mis-declarations).
If coverage < 90%, there's untyped content worth typing.

```
gbrain schema review-orphans --limit 50 --json
```

Untyped pages drilldown. Look for shared path prefixes (e.g. "12 of these
are under `research/papers/`") — those are candidates for a new type.

### Phase 3 — Propose (what types should the pack add?)

```
gbrain schema detect --json
```

Clusters pages by `source_path` and proposes candidate types. Heuristic
only (no LLM call).

```
gbrain schema suggest --json
```

LLM-refined candidates with confidence scores. Use the top-3 hit rate as
the signal for which to promote.

### Phase 4 — Apply (mutate the pack)

If the active pack is bundled (`gbrain-base` or `gbrain-recommended`),
fork it first:

```
gbrain schema fork gbrain-base mine
gbrain schema use mine
```

Then add the types one at a time:

```
gbrain schema add-type researcher \
  --primitive entity \
  --prefix people/researchers/ \
  --extractable \
  --expert
```

For complex multi-mutation refactors (e.g. add a type AND the link verb
that points to it), agents reaching this surface over MCP can use the
batched `schema_apply_mutations` op:

```jsonl
{"op": "add_type", "name": "researcher", "primitive": "entity", "prefix": "people/researchers/", "extractable": true, "expert_routing": true}
{"op": "add_type", "name": "paper", "primitive": "annotation", "prefix": "research/papers/", "extractable": true}
{"op": "add_link_type", "name": "authored", "inference": {"page_type": "researcher", "target_type": "paper"}}
```

Validate before sync:

```
gbrain schema lint --with-db
```

The `--with-db` flag opts into the 2 DB-aware rules
(`extractable_empty_corpus`, `mutation_count_anomaly`) that detect
mis-declared types you'd otherwise discover only at runtime.

### Phase 5 — Sync (backfill existing pages with the new types)

Dry-run first:

```
gbrain schema sync --json
```

Returns per-prefix `would_apply` counts + sample slugs. If the numbers
look right:

```
gbrain schema sync --apply
```

Chunked UPDATE in 1000-row batches; never wedges concurrent writers.
Idempotent on re-run (second `--apply` finds nothing to backfill).

### Phase 6 — Verify

```
gbrain schema stats --json
```

Coverage should be ≥95% now. Spot-check the new type:

```
gbrain whoknows "machine learning"
```

If `researcher` was declared `--expert`, results should include
researcher-typed pages. (The pack-aware wiring at the query path was
added in v0.40.6.0 — pre-v0.40.6 brains silently ignored custom
expert-routed types.)

### Phase 7 — Commit (preserve the change)

If the pack is in source control, commit:

```
cd ~/.gbrain/schema-packs/mine
git add pack.json
git commit -m "schema: add researcher + paper types + authored link"
git push
```

If the brain daemon is running (`gbrain serve --http`), other processes
pick up the change within 1 second (stat-mtime TTL gate in
loadActivePack — v0.40.6.0 closed the cross-process invalidation gap).

## Outputs

- Mutated pack file at `~/.gbrain/schema-packs/<name>/pack.{json,yaml}`.
- Audit row in `~/.gbrain/audit/schema-mutations-YYYY-Www.jsonl` per mutation.
- `pages.type` backfilled on matching rows after `sync --apply`.
- Query paths (`whoknows`, `find_experts`) now route through the new
  expert types.

## Contract

- **Inputs:** a natural-language request that names a type / prefix / link verb / flag change, OR the result of `gbrain schema review-orphans` showing untyped pages that need a new type.
- **Outputs:** mutated pack file at `~/.gbrain/schema-packs/<name>/pack.{json,yaml}` + an audit row in `~/.gbrain/audit/schema-mutations-YYYY-Www.jsonl` + (if `sync --apply` ran) backfilled `pages.type` on matching rows.
- **Side effects:** invalidates the in-process pack cache + the query cache for the source. Other processes pick up the change within 1 second (stat-mtime TTL).
- **Idempotency:** every primitive is idempotent. `add-alias`/`add-prefix` no-op on duplicate; `sync --apply` finds nothing to update on second run.
- **Trust:** CLI = local trust (no scope check). MCP = OAuth `admin` scope (write ops). Audit log captures `actor: mcp:<clientId8>` per mutation.
- **Atomicity:** every mutation is wrapped in `withMutation`'s atomic write (`.tmp + fsync + rename`) + per-pack `O_CREAT|O_EXCL` lock. Crash mid-write leaves the original file untouched.

## Anti-Patterns

- **Don't mutate `gbrain-base` or `gbrain-recommended`.** Fork first (`gbrain schema fork gbrain-base mine`). These are bundled packs; edits would be lost on upgrade. The mutation primitives refuse with `PACK_READONLY`.
- **Don't add a type for a directory you imported once for triage.** Pack types are permanent decisions; one-time imports are not. See `skills/conventions/schema-evolution.md` for the <20-pages-don't-pack-codify heuristic.
- **Don't add `--expert` to a type with no `path_prefixes`.** The `expert_routing_without_prefix` lint warns about this — expert-routed types with no prefix never match a put_page inference, so `whoknows` silently never surfaces them.
- **Don't prom