high-dim-intuition-rebuild

What this skill does

# High-Dim Intuition Rebuild

## Table of Contents
1. [Workflow](#workflow)
2. [The Five Big Lies of 3D Intuition](#the-five-big-lies-of-3d-intuition)
3. [Repair Patterns](#repair-patterns)
4. [Common Patterns](#common-patterns)
5. [Guardrails](#guardrails)
6. [Quick Reference](#quick-reference)

3D intuition is the user's most powerful asset for low-dim geometry — and their most dangerous liability for high-dim geometry. In 1000 dimensions, the unit ball is mostly empty, samples from any rotation-invariant distribution live on a thin shell, "nearest neighbor" stops being meaningful, and most of a hypercube's volume is in its corners. None of these are *intuitive* from 3D, and learners who don't know to expect them will reason wrongly about embeddings, latent spaces, and data clouds.

This skill is the surgical version of that repair: name the false intuition, demonstrate where it breaks, install the correct picture.

**Quick example (concentration of measure):**

> **Claim from learner:** "If I sample from a high-dimensional Gaussian, I should expect samples near the origin — that's where the density peaks."
>
> **The false 3D intuition:** In 2D or 3D, samples *do* cluster near the origin. The density peak there gives a strong gravitational pull on samples.
>
> **Why it breaks in high D:** Density is per unit volume. The amount of volume at radius r grows as r^(d−1). In high d, even though density falls off near r = √d (the "shell"), the *volume* available there grows fast enough that density × volume is maximized on the shell, not at the origin.
>
> **Correct picture:** In d dimensions, almost all the mass of a standard Gaussian is on a thin shell at radius √d. The 3D mental image of "Gaussian = blob centered at the origin" is *wrong* in high D; the right image is "Gaussian = thin spherical shell of radius √d, very little mass anywhere else".
>
> **Verify:** numpy: `np.linalg.norm(np.random.randn(10000, 1000), axis=1).mean() ≈ 31.6 ≈ √1000`. The samples really do all live at that radius.

## Workflow

Copy this checklist and track your progress:

```
Rebuild Progress:
- [ ] Step 1: Identify the misleading 3D intuition the learner is using
- [ ] Step 2: Name the high-dim phenomenon that contradicts it
- [ ] Step 3: Show *why* it breaks (the mechanism, not just the fact)
- [ ] Step 4: Install the correct high-dim picture
- [ ] Step 5: Verify with a numpy / sympy demonstration when possible
- [ ] Step 6: Generalize — what other 3D intuitions does this break?
```

**Step 1: Identify the misleading 3D intuition the learner is using**

Most high-dim confusions trace to one of five universal 3D intuitions (see [The Five Big Lies of 3D Intuition](#the-five-big-lies-of-3d-intuition) below). Diagnose which:

- "Samples cluster near the mean / mode." → concentration of measure.
- "Nearest neighbors are meaningful." → distance metric breakdown.
- "Random vectors point in random directions." → angle concentration.
- "A ball fills most of its bounding cube." → volume in corners.
- "I can losslessly compress this 1000-D vector to 10-D and back." → ignores intrinsic dimension / manifold hypothesis.

Sometimes the learner doesn't *state* the intuition; they just express confusion at a phenomenon ("why is cosine similarity always ~0?"). Reverse-engineer to find which intuition would have predicted the wrong answer.

**Step 2: Name the high-dim phenomenon that contradicts it**

Each false intuition has a corresponding true high-dim phenomenon. Name it explicitly:

- "Concentration of measure" — for any reasonable function on a high-dim sphere or Gaussian, the function's value is *almost constant* (concentrated around its mean).
- "Curse of dimensionality" — distance metrics lose discriminative power.
- "Angle concentration" — random unit vectors are nearly orthogonal.
- "Cube-corner dominance" — the unit hypercube has almost all its volume in the corners.
- "Manifold hypothesis" — real data lives on a low-dim manifold inside the high-dim ambient space.

Naming matters. The learner who can name the phenomenon can look it up later and recognize it in new contexts.

**Step 3: Show *why* it breaks (the mechanism, not just the fact)**

This is the load-bearing step. Don't just assert that the 3D intuition is wrong — show the mechanism.

For concentration of measure:
> "Density × volume is what determines where samples land. In d dimensions, a thin shell at radius r has volume proportional to r^(d-1). For Gaussian density e^(-r²/2), the product r^(d-1)·e^(-r²/2) is maximized at r = √(d−1) ≈ √d. In 3D, that's just √3 ≈ 1.7 — close to the origin, intuition holds. In 1000D, that's √1000 ≈ 31.6 — far from the origin, intuition collapses."

The mechanism explanation always involves *how does the dimension d enter the formula?* — and the answer reveals what fights what (density vs. volume, here).

**Step 4: Install the correct high-dim picture**

Replace the broken 3D picture with one that's correct in any dimension:

- Bad picture: "Gaussian = bell-shaped blob at the origin."
- Correct picture: "Gaussian in d dim = thin spherical shell at radius √d. The blob picture only works for d ≤ 3."

The new picture should be *visualizable* — use 1D or 2D analogies that *survive* the dimension change:

- For Gaussian: "imagine a thin spherical shell" — this is a 3D image but it's the *correct* 3D image.
- For nearly-orthogonal vectors: "imagine two random pencils on a desk; they have almost any angle equally likely. Now imagine that almost all angles are 90°. That's high-D."

**Step 5: Verify with a numpy / sympy demonstration when possible**

A 3-line numpy script is worth a thousand words of intuition repair. Examples:

```python
import numpy as np
# Concentration of measure
samples = np.random.randn(10000, 1000)
print(np.linalg.norm(samples, axis=1).mean())  # ≈ √1000 ≈ 31.6
print(np.linalg.norm(samples, axis=1).std())   # very small

# Angle concentration
v1 = np.random.randn(10000, 1000)
v2 = np.random.randn(10000, 1000)
cos = (v1 * v2).sum(axis=1) / (np.linalg.norm(v1, axis=1) * np.linalg.norm(v2, axis=1))
print(cos.mean(), cos.std())  # ≈ 0, ≈ 1/√1000
```

If you have Bash access, *run* the script and report the actual numbers. Concrete numbers anchor the new intuition.

**Step 6: Generalize — what other 3D intuitions does this break?**

Each of the Five Big Lies has cascading consequences. The user who learns about concentration of measure should also be told:
- Their mental model of Gaussian sampling is wrong (most samples ≠ near origin).
- Distance from origin is *not* a useful "outlier score" in high D — most samples are at the same distance.
- Reasoning about "the mode" or "the center of mass" of a high-dim distribution is suspect.
- Latent-space sampling for VAEs is more subtle than 3D intuition suggests.

The generalization step is what makes the rebuild *durable* — it teaches the user to flag *similar* misuses of 3D intuition in the future.

For one full rebuild per phenomenon, see [resources/phenomena.md](resources/phenomena.md). For numpy demonstration scripts, see [resources/demos.md](resources/demos.md).

## The Five Big Lies of 3D Intuition

Almost every high-dim confusion is a version of one of these.

### Lie 1: "Samples cluster near the mean."
**True in 3D:** Yes — samples from a Gaussian or uniform on a ball *do* concentrate near the mean.
**False in high D:** Samples concentrate on a thin *shell* at distance √d from the mean. The "near the mean" zone is empty.
**Phenomenon:** Concentration of measure.
**Why it matters:** Any reasoning about "typical" samples being "near the center" is wrong in high D. Latent-space interpolation, sampling, mode-seeking optimization all need rethinking.

### Lie 2: "Nearest neighbors are well-defined."
**True in 3D:** The nearest neighbor of a query is meaningfully closer than the others.
**False in high D:** The ratio of nearest-distance to farthest-distance approaches 1. *Every* point is approximately the same distance from any