compare-docs

What this skill does

# Semantic Document Comparison Command

**Task**: Compare two documents semantically: `{{arg1}}` vs `{{arg2}}`

**Goal**: Determine if documents contain the same semantic content AND preserve relationships (temporal, conditional, cross-document) despite different wording/organization.

**Method**: Enhanced claim extraction + relationship extraction + execution equivalence scoring

**⚠️ CRITICAL: Independent Extraction Required**

This command MUST extract claims from BOTH documents independently. NEVER:
- Pre-populate with "items to verify" or "improvements to check"
- Prime the extractor with knowledge of what changed between documents
- Use targeted confirmation instead of full extraction

Targeted validation (telling extractor what to look for) inflates scores by confirming
a checklist rather than independently discovering all claims.

---

## Overview

**Workflow**:
1. **Extract enhanced claims + relationships from Document A IN PARALLEL**
2. **Extract enhanced claims + relationships from Document B IN PARALLEL**
3. Compare claim sets AND relationship graphs (after both complete)
4. Calculate execution equivalence score (claim 40% + relationship 40% + graph 20%)
5. Report: shared/unique claims, preserved/lost relationships, warnings

**⚡ CRITICAL: Steps 1 and 2 MUST run in parallel (single message with two Task calls)**
- Extractions are completely independent (no cross-contamination risk)
- Running sequentially wastes time (~50% slower for no accuracy benefit)
- Step 3 waits for both to complete before comparing

**Key Insight**: Claim preservation ≠ Execution preservation. Documents can have identical claims but different execution behavior if relationships are lost.

**Reproducibility and Determinism**:

This command aims for high reproducibility but cannot guarantee perfect determinism due to LLM semantic judgment.

**Sources of Variance**:
1. **LLM Temperature** (±2-5% score variance if >0)
   - Mitigation: Use `temperature=0` in all Task calls (already specified above)
   - Expected with temp=0: ±0.5-1% residual variance
2. **Model Version** (±1-3% score drift across versions)
   - Mitigation: Pin exact model version (e.g., `"claude-opus-4-5-20251101"`)
   - Already specified in Task calls above
3. **Semantic Judgment** (±1-2% for boundary cases)
   - Claim similarity: "essentially the same" vs "slightly different"
   - Relationship matching: "same constraint" vs "subtly modified"
   - Inherent to semantic comparison, cannot be eliminated
4. **Claim Boundary Detection** (±0.5-1% for complex nested claims)
   - Conjunctions: Split into parts vs preserved as unit
   - Conditionals: Boundary of IF-THEN-ELSE scope
   - Minor variance in claim count (e.g., 191 vs 193)

**Expected Reproducibility**:
- **Same session, same documents**: ±0-1% (near-identical, small rounding differences)
- **Different sessions, temp=0, pinned model**: ±1-2% (good reproducibility)
- **Different sessions, temp>0**: ±3-7% (moderate variance)
- **Different model versions**: ±5-10% (significant drift possible)

**Best Practices for Consistency**:
- Always use `temperature=0` (already specified in Task calls)
- Pin model version if absolute consistency required across sessions
- Accept ±1-2% variance as inherent to semantic analysis
- Focus on score interpretation range (≥0.95, 0.85-0.94, etc.) not exact decimal

---

## Steps 1 & 2: Extract Claims + Relationships from BOTH Documents (IN PARALLEL)

**⚡ CRITICAL**: Invoke BOTH extraction agents in a single message with two Task tool calls.

**Why Parallel Execution**:
- ✅ **Safe**: Extractions are completely independent (no shared state)
- ✅ **Accurate**: No cross-contamination between Document A and B analysis
- ✅ **Faster**: ~50% time reduction (both extractions run simultaneously)
- ✅ **Required by Step 3**: Comparison waits for both anyway

**Agent Prompt Template** (use for BOTH documents):

**Agent Prompt**:
```
**SEMANTIC CLAIM AND RELATIONSHIP EXTRACTION**

**Document**: {{arg1}}

**Your Task**: Extract all semantic claims AND relationships from this document.

---

## Part 1: Claim Extraction

**What is a "claim"?**
- A requirement, instruction, rule, constraint, fact, or procedure
- A discrete unit of meaning that can be verified as present/absent
- Examples: "must do X before Y", "prohibited to use Z", "setting W defaults to V"

**Claim Types**:

1. **Simple Claims** (requirement, instruction, constraint, fact, configuration)
2. **Conjunctions**: ALL of {X, Y, Z} must be true
   - Markers: "ALL of the following", "both X AND Y", "requires all"
   - Example: "Approval requires: technical review AND budget review AND strategic review"
3. **Conditionals**: IF condition THEN consequence_true ELSE consequence_false
   - Markers: "IF...THEN...ELSE", "when X, do Y", "depends on"
   - Example: "IF attacker has monitoring THEN silent block ELSE network disconnect"
4. **Consequences**: Actions that result from conditions/events
   - Markers: "results in", "causes", "leads to", "enforcement"
   - Example: "Violating Step 1 causes data corruption (47 transactions affected)"
5. **Negations with Scope**: Prohibition with explicit scope
   - Markers: "NEVER", "prohibited", "CANNOT", "forbidden"
   - Example: "CANNOT run Steps 2 and 3 in parallel (data corruption risk)"

**Extraction Rules**:

1. **Granularity**: Atomic claims (cannot split without losing meaning)
2. **Completeness**: Extract ALL claims, including implicit ones if unambiguous
3. **Context**: Include minimal context for understanding
4. **Exclusions**: Skip pure examples, meta-commentary, table-of-contents

**Normalization Rules** (apply to all claim types):

1. **Tense**: Present tense ("create" not "created")
2. **Voice**: Imperative/declarative ("verify changes" not "you should verify")
3. **Synonyms**: Normalize common variations:
   - "must/required/mandatory" → "must"
   - "prohibited/forbidden/never" → "prohibited"
   - "create/establish/generate" → "create"
   - "remove/delete/cleanup" → "remove"
   - "verify/validate/check/confirm" → "verify"
4. **Negation**: Standardize ("must not X" → "prohibited to X")
5. **Quantifiers**: Normalize ("≥80%", "<100")
6. **Filler**: Remove filler words

---

## Part 2: Relationship Extraction

**Relationship Types to Extract**:

### 1. Temporal Dependencies (Step A → Step B)
**Markers**: "before", "after", "then", "Step N must occur after Step M", "depends on completing"
**Example**: "Step 3 (data migration) requires Step 2 (schema migration) to complete first"
**Constraint**: strict=true if order violation causes failure

### 2. Prerequisite Relationships (Condition → Action)
**Markers**: "prerequisite", "required before", "must be satisfied before"
**Example**: "All prerequisites (A, B, C) must be satisfied before Step 1"
**Constraint**: strict=true if prerequisite skipping causes failure

### 3. Hierarchical Conjunctions (ALL of X must be true)
**Markers**: "ALL", "both...AND...", "requires all", nested lists
**Example**: "Level 1: (A1 AND A2 AND A3) AND (B1 AND B2 AND B3)"
**Constraint**: all_required=true

### 4. Conditional Relationships (IF-THEN-ELSE)
**Markers**: "IF...THEN...ELSE", "when X, do Y", "depends on"
**Example**: "IF system powered on THEN memory dump ELSE disk removal"
**Constraint**: mutual_exclusivity=true for alternatives

### 5. Exclusion Constraints (A and B CANNOT co-occur)
**Markers**: "CANNOT run concurrently", "NEVER together", "mutually exclusive"
**Example**: "Steps 2 and 3 CANNOT run in parallel (data corruption risk)"
**Constraint**: strict=true if violation causes failure

### 6. Escalation Relationships (State A → State B under trigger)
**Markers**: "escalate to", "redirect to", "upgrade severity"
**Example**: "MEDIUM incident escalates to HIGH if privilege escalation possible"
**Constraint**: trigger condition explicit

### 7. Cross-Document References (Doc A → Doc B Section X)
**Markers**: "see Section X.Y", "defined in Document Z", "refer to"
**Example**: "Technical Architect (se