evo-memory

What this skill does

# Evo-Memory

A persistent learning layer that accumulates research knowledge across ideation and experimentation cycles. Maintains two memory stores and implements three evolution mechanisms that feed learned patterns back into future research.

## When to Use This Skill

- User has completed an `research-ideation` and needs to update Ideation Memory
- User has completed (or failed) an `experiment-pipeline` and needs to update memory
- User is starting a new research cycle and wants to load prior knowledge
- User asks about research memory, learned patterns, or cross-cycle knowledge
- User mentions "evo-memory", "update memory", "what worked before", "research history", "evolution"

## The Learning Layer

Research is iterative. Each cycle — from ideation through experimentation — generates knowledge that should inform the next cycle. Without persistent memory, every new project starts from scratch, repeating mistakes and rediscovering patterns.

Evo-memory solves this by maintaining two structured memory stores and three evolution mechanisms that extract, classify, and inject knowledge across cycles.

## Two Memory Stores

### Ideation Memory (M_I)

**Location**: `/memory/ideation-memory.md`

Records what you've learned about research DIRECTIONS — which areas are promising and which are dead ends.

**Two sections**:

| Section | What It Contains | Example Entry |
|---------|-----------------|---------------|
| Feasible Directions | Directions that showed promise in prior cycles | "Contrastive learning for few-shot classification — confirmed feasible, top-3 in tournament cycle 2" |
| Unsuccessful Directions | Directions that were tried and failed, with failure classification | "Autoregressive generation for real-time video — fundamental failure: latency constraint incompatible with autoregressive decoding" |

**Each entry records**: Direction name, one-sentence summary, evidence (which cycle, what results), classification (feasible / implementation failure / fundamental failure), date.

**How it's used**: `research-ideation` reads M_I at the start of Step 0. The paper uses embedding-based retrieval with cosine similarity, selecting the top-k_I most similar items (k_I=2 in experiments). Feasible directions from prior cycles can seed new tree branches. Unsuccessful directions are used during pruning — fundamental failures are pruned; implementation failures may be retried.

See [assets/ideation-memory-template.md](assets/ideation-memory-template.md) for the template.

### Experimentation Memory (M_E)

**Location**: `/memory/experiment-memory.md`

Records what you've learned about research STRATEGIES — which technical approaches and configurations work in practice.

The paper defines M_E as storing "reusable data processing and model training strategies." ESE jointly summarizes (i) a **data processing strategy** and (ii) a **model training strategy**. We extend this with two additional practical sections (architecture and debugging) for comprehensive coverage.

**Two core sections (from paper) + two practical extensions**:

| Section | Source | What It Contains | Example Entry |
|---------|--------|-----------------|---------------|
| Data Processing Strategies | Paper (core) | Preprocessing, augmentation, and data handling patterns | "For noisy sensor data: median filter before normalization reduces training instability by ~40%" |
| Model Training Strategies | Paper (core) | Hyperparameters, training tricks, and training schedules | "Learning rate warmup for 10% of steps prevents early divergence in transformer fine-tuning" |
| Architecture Strategies | Extension | Design choices, module configurations, and structural patterns | "Residual connections are critical for modules inserted deeper than 10 layers in transformers" |
| Debugging Strategies | Extension | Diagnostic patterns that resolved experiment failures | "When loss plateaus after 50% of training: check gradient norm — clipping threshold may be too aggressive" |

**Each entry records**: Strategy name, context (when to use this), evidence (which cycle, what results), generality (domain-specific or broadly applicable), date.

**How it's used**: `experiment-pipeline` reads M_E at the start of each cycle. The paper uses embedding-based retrieval with cosine similarity, selecting the top-k_E most similar items (k_E=1 in experiments). Relevant strategies from prior cycles inform hyperparameter choices, data processing decisions, and debugging approaches, reducing the number of attempts needed.

See [assets/experiment-memory-template.md](assets/experiment-memory-template.md) for the template.

## Three Evolution Mechanisms

### IDE — Idea Direction Evolution

**Trigger**: After `research-ideation` completes Step 5 and saves `/direction-summary.md` for Step 6.

**Purpose**: Extract promising research directions from the tournament results and store them in M_I for future cycles.

**Paper Prompt**: Use the IDE prompt from [references/paper-prompts.md](references/paper-prompts.md) as the primary extraction mechanism. Fill in `{user_goal}` from the original research direction and `{top_ranked_ideas}` from `/direction-summary.md`, then reason through the prompt step by step. The output (DIRECTION SUMMARY with Title, Core idea, Why promising, Requirements, Validation plan) feeds directly into the steps below.

**Process**:
1. Read current M_I from `/memory/ideation-memory.md`
2. Run the paper's IDE prompt (see above), reasoning through it step by step
3. For each direction in the prompt output, abstract it to a reusable level. "Attention-based feature selection for 3D point clouds" becomes "Cross-domain attention mechanisms for sparse data" — specific enough to be useful, abstract enough to transfer.
4. Check M_I for existing entries on similar directions. Update if exists, append if new.
5. If any previously "feasible" direction was found to be exhausted during this cycle, update its status.
6. Write an evolution report documenting what changed and why.

**Key principle**: Store directions, not ideas. A direction like "contrastive learning for structured data" can spawn many specific ideas across future cycles. A specific idea like "SimCLR with graph augmentations on molecular datasets" is too narrow to be reusable.

See [references/ide-protocol.md](references/ide-protocol.md) for the full process.

### IVE — Idea Validation Evolution

**Trigger** (two conditions, following the paper):
1. **Rule-based**: The engineer cannot find any executable code within the pre-defined budget at any stage — the code simply doesn't run.
2. **LLM-based**: Experiments complete but the proposed method performs worse than the baseline, as determined by analyzing the execution report W.

**Purpose**: Classify WHY the method failed and update M_I accordingly. This is the most critical evolution mechanism because it prevents future cycles from repeating dead-end directions.

**Paper Prompt**: Use the IVE prompt from [references/paper-prompts.md](references/paper-prompts.md) as the primary classification mechanism. Fill in `{research_proposal}` from `/research-proposal.md` and `{execution_report}` from the stage trajectory logs, then reason through the prompt step by step. The prompt classifies the failure as FAILED(NoExecutableWithinBudget), FAILED(WorseThanBaseline), or NOT_FAILED.

**After running the paper prompt**:
- **FAILED(NoExecutableWithinBudget)** → Implementation failure (retryable). Record as "retry with fixes" in M_I.
- **FAILED(WorseThanBaseline)** → Use the 5-question diagnostic below to distinguish implementation vs fundamental failure.
- **NOT_FAILED** → No IVE update needed.

**Five diagnostic questions** (for WorseThanBaseline cases):
1. Did any variant show partial success? (Yes → implementation failure)
2. Does the hypothesis hold for simpler problems? (No → fundamental failure)
3. Have related approaches succeeded in published work? (Yes → implementation failure)
4. Were failure patterns consistent acr