ln-100-documents-pipeline

What this skill does

> **Paths:** File paths (`references/`, `../ln-*`) are relative to this skill directory.

**Type:** L1 Top Orchestrator
**Category:** 1XX Documentation Pipeline

# Documentation Pipeline (Orchestrator)

This skill orchestrates the creation of a complete documentation system by invoking one L2 coordinator plus 3 L2 workers. The coordinator (ln-110) delegates to 5 L3 workers for project docs; other L2 workers handle reference, tasks, and test domains. Each component validates its own output, and the orchestrator owns the final docs-quality acceptance gate.

## Purpose

Top-level orchestrator that creates a complete project documentation system in one invocation. Chains ln-110 coordinator + ln-120/130/140 workers sequentially, then runs global cleanup (deduplication, orphan reporting, cross-link validation).

## Runtime Contract

**MANDATORY READ:** Load `references/coordinator_runtime_contract.md`, `references/docs_pipeline_runtime_contract.md`, `references/docs_generation_summary_contract.md`

Runtime family: `docs-pipeline-runtime`

Identifier:
- `docs-pipeline`

Phases:
1. `PHASE_0_CONFIG`
2. `PHASE_1_LEGACY_SCAN`
3. `PHASE_2_CONFIRMATION`
4. `PHASE_3_DELEGATE`
5. `PHASE_4_QUALITY_GATE`
6. `PHASE_5_CLEANUP`
7. `PHASE_6_SELF_CHECK`

Component summary contract:
- `ln-110`, `ln-120`, `ln-130`, `ln-140` write or return `docs-generation` envelopes
- `ln-100` consumes those summaries plus docs-quality checkpoints as orchestration SSOT

## Architecture

```
ln-100-documents-pipeline (L1 Top Orchestrator - this skill)
|- ln-110-project-docs-coordinator (L2 Coordinator)
|  |- ln-111-root-docs-creator (L3 Worker) -> 5 docs
|  |- ln-112-project-core-creator (L3 Worker) -> 3 docs
|  |- ln-113-backend-docs-creator (L3 Worker) -> 2 conditional
|  |- ln-114-frontend-docs-creator (L3 Worker) -> 1 conditional
|  `- ln-115-devops-docs-creator (L3 Worker) -> 1 always + 1 conditional
|- ln-120-reference-docs-creator (L2 Worker)
|- ln-130-tasks-docs-creator (L2 Worker)
`- ln-140-test-docs-creator (L2 Worker - optional)
```

## When to Use This Skill

This skill should be used when:
- Start a new IT project and need complete documentation system at once
- Use automated workflow instead of manually invoking multiple workers
- Create entire documentation structure (`AGENTS.md` canonical + `CLAUDE.md` import stub -> `docs/`) in one go
- Prefer orchestrated CREATE path over manual skill chaining
- Need automatic global cleanup (deduplication, orphaned files, consolidation)

**Alternative**: If you prefer granular control, invoke workers manually:
1. [ln-110-project-docs-coordinator](../ln-110-project-docs-coordinator/SKILL.md) - Root + Project docs (coordinates 5 L3 workers)
2. [ln-120-reference-docs-creator](../ln-120-reference-docs-creator/SKILL.md) - reference/ structure
3. [ln-130-tasks-docs-creator](../ln-130-tasks-docs-creator/SKILL.md) - tasks/README.md + kanban
4. [ln-140-test-docs-creator](../ln-140-test-docs-creator/SKILL.md) - tests/README.md (optional)

**MANDATORY READ:** Load `references/docs_quality_contract.md`.

**Note**: Each worker validates its own local output in Phase 2/3. The orchestrator still owns the final docs-quality manifest, verification, and repair loop before cleanup can continue.

## Workflow

The skill follows a 5-phase orchestration workflow: **Legacy Migration (optional)** -> User confirmation -> Invoke coordinator + workers sequentially -> Global cleanup -> Summary.

---

### Phase 0: Source Documentation Cleanup (OPTIONAL)

**Objective**: Detect non-canonical documentation, normalize any useful facts, and remove duplicate documentation paths before generation.

**Trigger**: Runs at pipeline start when the repo already contains documentation outside the canonical layout.

**Process**:

**0a. Non-Canonical Source Detection**:
- Scan project for non-standard documentation using patterns from `references/legacy_detection_patterns.md`:
  - **Root .md files**: `ARCHITECTURE.md`, `REQUIREMENTS.md`, `STACK.md`, `API.md`, `DATABASE.md`, `DEPLOYMENT.md`
  - **Non-canonical folders**: `documentation/`, `doc/`, `wiki/`, or `docs/` with wrong structure
  - **README.md sections**: `## Architecture`, `## Tech Stack`, `## Requirements`, etc.
  - **CONTRIBUTING.md sections**: `## Development`, `## Code Style`, `## Coding Standards`
- Build `source_manifest`: list of { path, detected_type, target_doc, confidence }
- If no non-canonical docs are found -> skip to Phase 1

**0b. Fact Extraction**:
- For each detected source file:
  - Parse markdown structure (headers, lists, code blocks)
  - Apply type-specific extractor (**MANDATORY READ:** Load `references/legacy_detection_patterns.md`):
    - `architecture_extractor` -> { layers[], components[], diagrams[] }
    - `requirements_extractor` -> { functional[], non_functional[] }
    - `tech_stack_extractor` -> { frontend, backend, database, versions }
    - `principles_extractor` -> { principles[], anti_patterns[] }
    - `api_spec_extractor` -> { endpoints[], authentication }
    - `database_schema_extractor` -> { tables[], relationships[] }
    - `runbook_extractor` -> { prerequisites[], install_steps[], env_vars[] }
    - `infrastructure_extractor` -> { servers[], domains[], ports[], services[], artifacts{}, cicd{} }
  - Score content quality (0.0-1.0)
- Store results in `source_notes`

**0c. User Confirmation**:
- Display detected non-canonical files:
  ```
  Non-Canonical Documentation Detected:

  | File | Type | Confidence | Target |
  |------|------|------------|--------|
  | README.md (## Architecture) | architecture | HIGH | docs/project/architecture.md |
  | docs/ARCHITECTURE.md | architecture | HIGH | docs/project/architecture.md |
  | CONTRIBUTING.md (## Development) | principles | MEDIUM | docs/principles.md |

  Cleanup Options:
  1. NORMALIZE (recommended): Extract facts -> Archive -> Remove duplicate sources
  2. ARCHIVE ONLY: Backup without extraction
  3. SKIP: Leave non-canonical docs as-is (may cause duplication)

  Choose option (1/2/3): _
  ```
- If user selects "1" (NORMALIZE):
  - Optional: "Review extracted facts before folding them into the canonical context? (yes/no)"
  - Confirm: "Proceed with cleanup and archive non-canonical files?"
- If user selects "2" (ARCHIVE ONLY):
  - Confirm: "Archive non-canonical files to .archive/? Content will NOT be normalized."
- If user selects "3" (SKIP):
  - Warn: "Non-canonical files will remain. This may cause duplication issues."
  - Proceed to Phase 1

**0d. Backup and Archive**:
- Create `.archive/source-docs-{timestamp}/` directory
- Structure:
  ```
  .archive/
  `- source-docs-YYYY-MM-DD-HHMMSS/
      |- README_cleanup.md          # Rollback instructions
      |- original/                  # Exact copies of non-canonical files
      |  |- README.md
      |  |- ARCHITECTURE.md
      |  `- documentation/
      `- extracted/                 # Extracted facts (for reference)
          |- architecture_content.md
          `- principles_content.md
  ```
- Copy all detected source files to `original/`
- Save extracted facts to `extracted/`
- Generate `README_cleanup.md` with rollback instructions

**0e. Context Normalization**:
- Build `SOURCE_DOC_NOTES` from extracted facts:
  ```json
  {
    "SOURCE_DOC_NOTES": {
      "architecture_notes": { "sections": [...], "diagrams": [...] },
      "requirements_notes": { "functional": [...] },
      "principles_notes": { "principles": [...] },
      "tech_stack_notes": { "frontend": "...", "backend": "..." },
      "api_notes": { "endpoints": [...] },
      "database_notes": { "tables": [...] },
      "runbook_notes": { "install_steps": [...] },
      "infrastructure_notes": { "servers": [...], "domains": [...], "ports": {} }
    }
  }
  ```
- Merge extracted facts into the coordinator input as supporting evidence only
- Workers consume normalized context fields and current source facts directly; there is no legacy-only branch
- Priority order: **Current source facts + normalized project documentati