generating-lattice

What this skill does

# Generating Lattice

Generate and maintain `lat.md/` knowledge graphs — structured, cross-referenced
markdown anchored to source code symbols. The output is validated by the `lat`
CLI, which catches documentation drift whenever code or docs change.

**The core mechanism is bidirectional linking:**

- **Top-down** (docs → code): `[[src/auth.js#getValidToken]]` wiki links in
  `lat.md/` files point to specific symbols in source. When a symbol is renamed
  or deleted, `lat check` catches the broken link.
- **Bottom-up** (code → docs): `// @lat: [[auth#Token Refresh]]` comments in
  source point back to the concept they implement. When a section is renamed or
  removed, `lat check` catches the dangling reference.

Without both directions, the lattice is a static essay that drifts silently.
With both, `lat check` enforces consistency.

**Dependency:** Requires **mapping-codebases** skill. The structural maps
(`_MAP.md` files) are the input that makes LLM-assisted authoring
token-efficient — read maps to understand API surfaces, then selectively read
source files only where design rationale lives.

## Installation

```bash
npm install -g lat.md
lat --version  # verify (requires Node.js 22+)
```

The `lat` CLI provides `lat check` (validation), `lat search` (semantic
search), `lat section` (browsing), `lat refs` (reference lookup), and `lat mcp`
(agent integration).

## Generation Pipeline

### Phase 1: Structural Scan

Ensure `_MAP.md` files exist. Generate them if missing:

```bash
# Install mapping-codebases dependencies
uv venv /home/claude/.venv
uv pip install tree-sitter-language-pack --python /home/claude/.venv/bin/python

# Generate maps
/home/claude/.venv/bin/python /mnt/skills/user/mapping-codebases/scripts/codemap.py /path/to/repo \
  --skip tests,.github,node_modules,vendor
```

Read the root `_MAP.md` first for high-level orientation, then drill into
subdirectory maps. The maps are the symbol inventory — they tell you what exists
and where, so you can write source code links without reading every file.

### Phase 2: Selective Source Reading

Maps reveal the API surface (exports, signatures, line numbers) without reading
full files. Read source selectively to understand design rationale:

**Read fully:** Files with complex algorithms, business logic, configuration
constants, data schemas. The maps identify these by export density and file
size.

**Read partially:** Header comments and constant blocks in large files.

**Skip:** Files where the map signature tells the full story (thin wrappers,
simple CRUD, format utilities).

### Phase 3: Generate Anchored Sections

Create `lat.md/` directory and generate markdown files. **Every section must be
anchored to source code symbols** — this is what makes `lat check` catch drift.

#### Section Structure Rules

- Every section MUST have a leading paragraph ≤250 characters (excluding
  `[[wiki link]]` content). This is the section's identity in search results.
- `lat check` validates this rule.

```markdown
# Good Section

Brief overview anchored to [[src/auth.js#refreshToken]] and [[src/api.js#fetchWithRetry]].

More detail can follow in subsequent paragraphs.
```

#### Source Code Links (the top-down anchors)

Use `[[src/path/file.ext#symbolName]]` wiki links to anchor documentation to
specific symbols. Supported extensions: `.ts`, `.tsx`, `.js`, `.jsx`, `.py`,
`.rs`, `.go`, `.c`, `.h`.

```markdown
## Token Refresh

OAuth token lifecycle managed by [[src/auth.js#getValidToken]] with automatic
refresh via [[src/auth.js#refreshToken]]. The refresh window
([[src/auth.js#REFRESH_THRESHOLD_MS]]) triggers proactive renewal before expiry.
```

**Symbol granularity matters.** Link to the specific function, class, constant,
or method — not just the file. `[[src/auth.js#refreshToken]]` is useful;
`[[src/auth.js]]` tells you nothing that the filename didn't already say.

For class methods: `[[src/server.ts#App#listen]]`. For Rust impl methods:
`[[src/lib.rs#Greeter#greet]]`. For Python: `[[lib/utils.py#parse_args]]`.

#### Section-to-Section Links

Use `[[file#Section#Subsection]]` or short form `[[file#Section]]` when the
file stem is unique:

```markdown
The sync pipeline ([[data#Sync Pipeline]]) uses the same token refresh
mechanism described in [[auth#Token Refresh]].
```

#### Conceptual Grouping

Organize by domain concept, not file structure. A `data.md` file might reference
`db.js`, `sync.js`, and `demo.js` because they're all part of the data layer.
The `_MAP.md` files already provide file-level structure; `lat.md/` adds the
semantic layer explaining WHY things connect.

#### Index File

`lat.md/lat.md` must contain a bullet list of all files with one-sentence
descriptions using wiki links. `lat check --index` validates completeness:

```markdown
- [[architecture]] — System design, OAuth flow, routing, signals
- [[data]] — IndexedDB schema, sync pipeline, demo mode
- [[awards]] — Award computation engine, data quality rules
```

Subdirectories need their own index: `lat.md/api/api.md`.

### Phase 4: Add Back-Links in Source Code

Add `// @lat:` or `# @lat:` comments in source code pointing back to `lat.md/`
sections. **This is not optional** — back-links are what make `lat check`
detect when source code changes break documentation.

Run the back-link helper to identify where annotations are needed:

```bash
python3 SKILL_DIR/scripts/suggest_backlinks.py /path/to/repo
```

This parses all `[[src/...]]` wiki links from `lat.md/` files, looks up
referenced symbols in `_MAP.md` files for O(1) line number resolution, and
suggests `@lat:` comment placements. Requires `_MAP.md` files from Phase 1.

Comment syntax by language:
- JS/TS/Rust/Go/C: `// @lat: [[section#Subsection]]`
- Python: `# @lat: [[section#Subsection]]`

Place one `@lat:` comment per section reference, at the relevant code location —
not at the top of the file:

```javascript
// @lat: [[auth#Token Refresh]]
async function refreshToken(token) { ... }
```

To auto-apply suggestions (review the output first):

```bash
python3 SKILL_DIR/scripts/suggest_backlinks.py /path/to/repo --apply
```

### Phase 4b: Annotate Maps with Lattice Cross-References

After backlinks exist in source, annotate `_MAP.md` files so someone browsing
a code map can find the lat.md section that explains WHY a module is designed
the way it is:

```bash
python3 SKILL_DIR/scripts/annotate_maps.py /path/to/repo
```

This scans `@lat:` comments in source and adds `> Documented in:` lines to
`_MAP.md` file headers. Idempotent — safe to re-run after regenerating maps.

Example output in `_MAP.md`:
```markdown
### auth.js
> Documented in: [[auth#Token Refresh]], [[auth#Server]]
> Imports: `express`
- **refreshToken** (f) `(token)` :15
```

#### require-code-mention for Critical Sections

For sections where bidirectional traceability is essential (test specs,
invariants, key business rules), add frontmatter:

```yaml
---
lat:
  require-code-mention: true
---
```

This makes `lat check` fail if ANY leaf section in the file lacks a
corresponding `@lat:` comment in the codebase. Use for test spec files where
every test must trace to its specification.

### Phase 5: Validate

```bash
cd /path/to/repo
lat check
```

This runs ALL checks:
- **md** — every `[[wiki link]]` in `lat.md/` resolves to a real section or
  source symbol
- **code-refs** — every `@lat:` comment in source points to a real section; and
  every leaf section in `require-code-mention` files is referenced by code
- **sections** — every section has a leading paragraph ≤250 chars
- **index** — every directory in `lat.md/` has a complete index file

**All four must pass.** Fix errors iteratively until `lat check` is clean.

### Phase 6: Agent Integration

Set up files that make coding agents maintain the lattice automatically.

**Cache exclusion** — create `lat.md/.gitignore`:
```
.cache/
```

**Agent instructions** — append to `CLAUDE.md` (or `AGENTS.md`) using markers:

```markdown
%% lat:begin %%
# Before starting wo