ebook-analysis

What this skill does

# Ebook Analysis: Non-Fiction Knowledge Extraction

You analyze ebooks to extract knowledge with full citation traceability. This skill supports two complementary extraction modes:

1. **Concept Extraction** - Extract ideas classified by abstraction (principle → tactic)
2. **Entity Extraction** - Extract named things (studies, researchers, frameworks, anecdotes) that persist across books

## Core Principle

**Every extraction must be traceable to its exact source.** Citation traceability is non-negotiable. Extract less with full provenance rather than more without it.

---

## Two Extraction Modes

### Mode 1: Concept Extraction
For extracting IDEAS organized by abstraction level.

**Use when:** Analyzing a book for transferable ideas, building a concept taxonomy, understanding how abstract principles relate to concrete tactics.

**Output:** JSON files (analysis.json, concepts.json)

**Example:** "Spaced repetition improves retention" is a MECHANISM at Layer 2.

### Mode 2: Entity Extraction
For extracting NAMED THINGS that can be cross-referenced across books.

**Use when:** Building a knowledge base where the same study, researcher, or framework appears in multiple books. The goal is entity resolution—recognizing that "Hogarth's framework" in Range is the same as "kind/wicked environments" mentioned elsewhere.

**Output:** Markdown files in knowledge base structure

**Example:** "Kind vs Wicked Environments" is a FRAMEWORK by Robin Hogarth.

### Choosing a Mode

| If you want to... | Use Mode |
|-------------------|----------|
| Understand a book's argument structure | Concept Extraction |
| Build a reference library across books | Entity Extraction |
| Create actionable takeaways | Concept Extraction |
| Track what researchers say across sources | Entity Extraction |
| Both | Run both modes sequentially |

---

## Entity Extraction Mode (Detailed)

### Entity Types

| Type | What It Captures | Example |
|------|------------------|---------|
| **study** | Research findings, experiments, data | Flynn Effect, Marshmallow Test |
| **researcher** | People and their contributions | Anders Ericsson, Robin Hogarth |
| **framework** | Mental models, taxonomies, systems | Kind vs Wicked, Desirable Difficulties |
| **anecdote** | Stories used to illustrate points | Tiger vs Roger, Challenger Disaster |
| **concept** | Ideas that aren't frameworks | Cognitive entrenchment, Match quality |

### Extended Entity Type Guidance

Some entities don't fit cleanly into the five types. Guidelines:

| Entity Kind | Use Type | Rationale |
|-------------|----------|-----------|
| **Simulations/Games** (Superstruct, EVOKE) | anecdote | Illustrative events, even if hypothetical |
| **Institutions** (IFTF, WEF) | researcher | Organizations contribute ideas like individuals |
| **Historical events** (Challenger disaster) | anecdote | Stories that illustrate principles |
| **Hypothetical scenarios** | anecdote | Future scenarios from books like Imaginable |
| **Thought experiments** | framework | If systematic; otherwise concept |

**When uncertain:** Default to `anecdote` for narratives/events, `concept` for ideas, `framework` for systematic methods.

### Author-as-Subject Pattern

When the book's author is also a significant entity (e.g., Jane McGonigal in Imaginable):

**Create a researcher entity if:**
- Author has notable prior work or institutional affiliation
- Author appears in Wikipedia or other reference sources
- Author's background/credentials are relevant to understanding the book
- Other books in your collection might reference them

**Skip if:**
- Author is primarily known only for this book
- No external sources to verify/enrich the entity

**Template addition for author-subjects:**
```markdown
## Note
This researcher is the author of [Book] in our collection. Their frameworks and concepts are documented separately.
```

### Entity File Template

```markdown
# [Entity Name]
**Type:** study | researcher | framework | anecdote | concept
**Status:** stub | partial | solid | authoritative
**Last Updated:** YYYY-MM-DD
**Aliases:** alias1, alias2, alias3

## Summary
[2-3 sentence synthesized understanding]

## Key Findings / What It Illustrates
1. [Claim or finding with source]
   — Source: [Book], Ch.[X]

2. [Another claim]
   — Source: [Book], Ch.[X]

## Key Quotes
> "Quotable text here."

> "Another memorable quote."

## Sources in Collection
| Book | Author | How It's Used | Citation |
|------|--------|---------------|----------|
| Range | Epstein | [Role in book] | Ch.X |

## Sources NOT in Collection
- [Book that would enrich this entity]

## Related Entities
- [Other Entity](../type/other-entity.md) - Relationship description

## Open Questions
- [What we don't yet know]
```

### Knowledge Base Structure

```
/knowledge/
├── _index.md                    # Master registry
├── _entities.json               # Searchable index (generated)
│
├── nonfiction/
│   ├── _index.md                # Domain index
│   ├── _[book]-quotes.md        # Book-specific quotes file
│   ├── studies/
│   │   ├── flynn-effect.md
│   │   └── chase-simon-chunking.md
│   ├── researchers/
│   │   ├── hogarth-robin.md
│   │   └── tetlock-philip.md
│   ├── frameworks/
│   │   ├── kind-vs-wicked-environments.md
│   │   └── desirable-difficulties.md
│   ├── anecdotes/
│   │   ├── tiger-vs-roger.md
│   │   └── challenger-disaster.md
│   └── concepts/
│       ├── cognitive-entrenchment.md
│       └── match-quality.md
│
├── cooking/                     # Domain-specific structure
│   ├── techniques/
│   ├── ingredients/
│   └── equipment/
│
└── technical/
    ├── patterns/
    └── technologies/
```

### Quotes Extraction

Quotable quotes are a distinct extraction type. For each book, create a quotes file:

**File:** `_[book-slug]-quotes.md`

**Structure:**
```markdown
# Quotable Quotes from [Book Title]
**Author:** [Author]
**Last Updated:** YYYY-MM-DD

## On [Theme 1]
> "Quote text here."

> "Another quote on same theme."

## On [Theme 2]
> "Quote on different theme."
```

**What makes a good quote:**
- Memorable phrasing that captures a key insight
- Self-contained (understandable without context)
- Surprising or counterintuitive formulation
- Useful for presentations, writing, or reference

### Entity Extraction Workflow

1. **Scan book** - Read through identifying named studies, researchers, frameworks, illustrative stories
2. **Check existing entities** - Use `kb-resolve-entity.ts` to see if entity already exists
3. **Create or update** - New entity → create file; existing → add as source
4. **Add quotes** - Extract memorable quotes to quotes file
5. **Cross-link** - Add Related Entities sections
6. **Regenerate index** - Run `kb-generate-index.ts`

### Entity Extraction States (KB0-KB5)

| State | Symptoms | Intervention |
|-------|----------|--------------|
| **KB0** | No knowledge base | Create directory structure |
| **KB1** | Structure exists, no entities | Begin extraction |
| **KB2** | Extracting from book | Create entity files |
| **KB3** | Entities created, not linked | Add Related Entities |
| **KB4** | Linked, no index | Run kb-generate-index.ts |
| **KB5** | Complete for this book | Proceed to next book |

### Cross-Book Synthesis Workflow

**Triggered when:** 2+ books have been extracted to the knowledge base.

**Goals:**
1. Find entities that appear in multiple books
2. Identify conceptual connections between books
3. Surface contradictions or complementary perspectives
4. Update entity files with multi-source synthesis

**Process:**

1. **Entity overlap detection**
   ```bash
   # Find entities with 2+ sources
   grep -l "Sources in Collection" knowledge/nonfiction/**/*.md | \
     xargs grep -l "| .* | .* |" | head -20
   ```
   Or manually review entities updated with new source.

2. **Conceptual connection mapping**
   - Compare frameworks across books (e.g., Range's "wicked environments" ↔ Imaginable's "futures thinking")
   - Identify shared researchers (e.g., Tetlock a