deep-dive-analysis

What this skill does

# Deep Dive Analysis Skill

## Overview

This skill combines **mechanical structure extraction** with **Claude's semantic understanding** to produce comprehensive codebase documentation. Unlike simple AST parsing, this skill captures:

- **WHAT** the code does (structure, functions, classes)
- **WHY** it exists (business purpose, design decisions)
- **HOW** it integrates (dependencies, contracts, flows)
- **CONSEQUENCES** of changes (side effects, failure modes)

## Language Support

| Language | Extensions | Structural extraction | Comment rewriting |
|---|---|---|---|
| Python | `.py`, `.pyi` | stdlib `ast` (always available) | `#` line + docstrings |
| Java | `.java` | tree-sitter (preferred) or regex | `//`, `/* */`, Javadoc `/** */` |
| JavaScript | `.js`, `.mjs`, `.cjs`, `.jsx` | tree-sitter (preferred) or regex | `//`, `/* */`, JSDoc `/** */` |
| TypeScript | `.ts`, `.tsx`, `.mts`, `.cts` | tree-sitter (preferred) or regex; adds interfaces, enums, type aliases | `//`, `/* */`, JSDoc `/** */` |
| SQL | `.sql`, `.ddl`, `.dml` | regex DDL (tables, views, indexes, sequences, types, functions, procedures, triggers) | `--`, `/* */` |
| PL/SQL (Oracle) | `.pks`, `.pkb`, `.plsql`, `.pls`, `.pck`, `.prc`, `.fnc`, `.trg` | regex (packages, package bodies, type bodies, cursors, exceptions, %TYPE/%ROWTYPE references) | `--`, `/* */` |
| Rust | `.rs` | tree-sitter (preferred) or regex; structs, enums, traits, impls (with `Trait for Type` naming), mods, unions, type aliases | `//`, `/* */`, rustdoc `///` / `//!` / `/** */` / `/*! */` |

`.sql` files are disambiguated against PL/SQL by inspecting content for Oracle-specific markers (`CREATE OR REPLACE PACKAGE`, `DBMS_OUTPUT`, `%TYPE`, `%ROWTYPE`, `UTL_FILE`, `PRAGMA AUTONOMOUS`, etc.). PostgreSQL `plpgsql` is correctly classified as SQL.

## Prerequisites

The scripts are designed to work **out of the box** with just the Python stdlib. Tree-sitter is optional and improves accuracy for Java / JavaScript / TypeScript:

```bash
# Optional: install for higher-fidelity parsing
pip install -r scripts/requirements.txt
# or
uv pip install -r scripts/requirements.txt
```

What changes when tree-sitter is installed:

- **Java**: nested classes, generic type parameters, annotations, multi-line declarations parsed correctly. Without it, the regex fallback still finds top-level classes, methods, imports, and constants.
- **JavaScript / TypeScript**: arrow functions in object/class properties, decorators, template literals, JSX elements parsed correctly. Without it, the regex fallback handles top-level declarations, ES6 `import`/`export`, and CommonJS `require`.
- **Rust**: lifetimes, generic bounds (`where` clauses), impl blocks with trait bounds, attribute macros parsed correctly. Without it, the regex fallback still finds top-level fns, structs/enums/traits/impls/mods, use declarations, and UPPER_CASE constants.
- **Python / SQL / PL-SQL**: no change. Python always uses stdlib `ast`; SQL/PL-SQL always use the regex DDL extractor.

The active parser is reported in `ParseResult.notes` and in the CLI output: `parser=stdlib-ast`, `parser=tree-sitter`, or `parser=regex-fallback`.

### Capabilities

**Mechanical Analysis (Scripts):**
- Extract code structure (classes, functions, imports)
- Map dependencies (internal/external)
- Find symbol usages across the codebase
- Track analysis progress
- Classify files by criticality

**Semantic Analysis (Claude AI):**
- Recognize architectural and design patterns
- Identify red flags and anti-patterns
- Trace data and control flows
- Document contracts and invariants
- Assess quality and maintainability

**Documentation Maintenance:**
- Review and maintain documentation (Phase 8)
- Fix broken links and update navigation indexes
- Analyze and rewrite code comments (antirez standards)

**Use this skill when:**
- Analyzing a codebase you're unfamiliar with
- Generating documentation that explains WHY, not just WHAT
- Identifying architectural patterns and anti-patterns
- Performing code review with semantic understanding
- Onboarding to a new project

## Prerequisites

This skill is invoked by the `/deep-dive-analysis` command. The command creates and manages state automatically in `.deep-dive/` under the target directory:

1. **`.deep-dive/state.json`** -- phase tracking (auto-created by the command)
2. **`.deep-dive/<phase-number>-<name>.md`** -- per-phase output documents

The legacy standalone flow using `analysis_progress.json` and `DEEP_DIVE_PLAN.md` at project root is no longer the primary path -- prefer invoking `/deep-dive-analysis <target>`.

## CRITICAL PRINCIPLE: ABSOLUTE SOURCE OF TRUTH

> **THE DOCUMENTATION GENERATED BY THIS SKILL IS THE ABSOLUTE AND UNQUESTIONABLE SOURCE OF TRUTH FOR YOUR PROJECT.**
>
> **ANY INFORMATION NOT VERIFIED WITH IRREFUTABLE EVIDENCE FROM SOURCE CODE IS FALSE, UNRELIABLE, AND UNACCEPTABLE.**

### Mandatory Rules (VIOLATION = FAILURE)

1. **NEVER** document anything without reading the actual source code first
2. **NEVER** assume any existing documentation, comment, or docstring is accurate
3. **NEVER** write documentation based on memory, inference, or "what should be"
4. **ALWAYS** derive truth EXCLUSIVELY from reading and tracing actual code
5. **ALWAYS** provide source file + qualified symbol name for every technical claim
6. **ALWAYS** verify state machines, enums, constants against actual definitions
7. **TREAT** all pre-existing docs as unverified claims requiring validation
8. **MARK** any unverifiable statement as `[UNVERIFIED - REQUIRES CODE CHECK]`
9. **USE** qualified symbol names in markers (`file.py::Class.method`), never line numbers -- line numbers break on any edit

See `references/analysis-templates.md` for the full verification trust model, temporal purity principle, and documentation status markers.

## Output Usage Guide

After analysis completes, consult the right file for your task:

| Your Task | Start With | Also Check |
|-----------|-----------|------------|
| Onboarding / understanding the project | 07-final-report, 01-structure | 04-semantics |
| Writing new feature | 01-structure (Where to Add), 02-interfaces | 04-semantics |
| Fixing a bug | 03-flows, 05-risks | 01-structure |
| Refactoring | 01-structure, 04-semantics, 05-risks | 03-flows |
| Code review | 02-interfaces, 05-risks | 06-documentation |
| Updating documentation | 06-documentation, 04-semantics | 02-interfaces |

## Forbidden Files

The analysis NEVER reads or includes contents from sensitive files: `.env`, `.env.*`, `credentials.*`, `secrets.*`, `*.pem`, `*.key`, `*.p12`, `*.pfx`, `id_rsa*`, `id_ed25519*`, `.npmrc`, `.pypirc`, `.netrc`, or any file containing API keys, passwords, or tokens. If encountered, note file existence only - never quote contents.

## Available Commands

### 1. Analyze Single File

```bash
# Python
python .claude/skills/deep-dive-analysis/scripts/analyze_file.py \
  --file src/utils/circuit_breaker.py \
  --output-format markdown

# Java
python .claude/skills/deep-dive-analysis/scripts/analyze_file.py \
  --file src/main/java/com/example/UserService.java

# TypeScript
python .claude/skills/deep-dive-analysis/scripts/analyze_file.py \
  --file src/services/auth.ts

# SQL / PL-SQL
python .claude/skills/deep-dive-analysis/scripts/analyze_file.py \
  --file migrations/0042_users.sql

python .claude/skills/deep-dive-analysis/scripts/analyze_file.py \
  --file packages/user_pkg.pkb
```

**Parameters:**
- `--file` / `-f`: Relative path to file - **REQUIRED**. Any supported extension (see Language Support table).
- `--output-format` / `-o`: Output format (json, markdown, summary) - default: summary
- `--find-usages` / `-u`: Find all usages of exported symbols - default: false
- `--update-progress` / `-p`: Update analysis_progress.json - default: false

### 2. Check Progress

```bash
python .claude/skills/deep-dive-analysis/scripts/check_progress.py \
  --phase 1 --status pending
```

### 3. Find Usages

```bash
python .claude/skills/d