data-driven-design

What this skill does

# Data-Driven Design

## When to Activate

Use this skill when:

- Setting up analytics infrastructure or telemetry pipelines
- Designing event taxonomies and instrumentation schemas
- Interpreting player behavior data or building hypotheses from metrics
- Running or evaluating A/B tests
- Building dashboards for design, executive, or live ops audiences
- Evaluating feature impact post-launch
- Making design decisions where quantitative evidence should inform the choice
- Diagnosing retention drops, progression bottlenecks, or economy imbalances
- Transitioning from playtest-phase observation to live data collection

---

## Core Principle: Data-Informed, Not Data-Driven

Data tells you **what** is happening. Design judgment tells you **why** and **what to do** about it.

The distinction matters. "Data-driven" implies the data decides. "Data-informed" means the data
contributes evidence to a decision that also weighs design intent, player experience goals, and
creative vision. Numbers without context are noise.

**Correlation ≠ causation** is the most violated principle in game analytics. Players who buy
the battle pass also play more — but forcing battle pass prompts on casual players won't make
them play more. The causation runs the other direction: engaged players buy, not the reverse.

Guidelines for healthy data use:

- Always ask "what would change our decision?" before collecting data
- Treat metrics as symptoms, not diagnoses — investigate before acting
- Preserve design intent; use data to refine execution, not replace vision
- When data and intuition conflict, dig deeper rather than defaulting to either
- Quantitative data says what happened; qualitative data (playtests, feedback) says why

---

## Telemetry Design

Good analytics start with good instrumentation. What you track and how you structure it
determines what questions you can answer later.

### Event Taxonomy

Organize events into three categories:

| Category | Description | Examples |
|---|---|---|
| **Player actions** | What the player chose to do | Equipped item, started quest, used ability, opened menu |
| **Game state events** | What the system did | Enemy spawned, loot dropped, difficulty adjusted, match started |
| **Session events** | Lifecycle markers | Session start, session end, app backgrounded, crash |

### Event Schema

Every event needs a minimum set of context fields:

| Field | Purpose | Notes |
|---|---|---|
| `timestamp` | When it happened | UTC, millisecond precision |
| `player_id` | Who did it | Persistent across sessions |
| `session_id` | Which play session | Links events within a session |
| `event_type` | What happened | From your taxonomy |
| `context` | Where in the game | Level, menu, match phase |
| `platform` | Device/OS | Segments behavior by hardware |
| `build_version` | Which version | Essential for before/after comparisons |

### Instrumentation Principles

1. **Track decision points, not just outcomes.** Knowing a player failed a level is less useful
   than knowing they attempted it 4 times, tried 2 different loadouts, and quit after the third
   attempt without the fourth completing.

2. **Track absence.** What players *don't* do is often more revealing than what they do. If a
   feature exists and nobody uses it, that's a signal. Instrument feature surfaces so you can
   measure both engagement and non-engagement.

3. **Track sequences, not just counts.** "Players used the shop 3 times" is less useful than
   "players opened the shop, browsed weapons, closed without buying, returned 10 minutes later,
   and purchased." Order and timing reveal intent.

4. **Include enough context to segment later.** You will always want to slice data by dimensions
   you didn't anticipate. Err toward including context fields rather than minimizing schema width.

### Data Volume Management

Not everything is worth tracking at full fidelity:

| Event Frequency | Strategy | Example |
|---|---|---|
| Per-frame (60/sec) | Sample at intervals or aggregate | Player position, camera angle |
| Per-second | Aggregate into summary events | DPS output, resource generation |
| Per-action (variable) | Full capture with context | Ability use, item purchase |
| Per-decision (rare) | Full capture with rich context | Build choice, quest selection |
| Per-session (1-2x) | Full capture | Session start/end, matchmaking |

The cost of storing high-frequency events rarely justifies the analytical value. Aggregate
per-frame data into periodic snapshots (every 5-10 seconds) unless you have a specific
analytical need for higher resolution.

---

## Key Metrics Framework

Organize metrics by what question they answer. Every metric listed here includes healthy
ranges as general baselines — actual targets vary by genre, platform, and business model.

### Engagement Metrics

| Metric | What It Measures | Healthy Range | Warning Signal |
|---|---|---|---|
| DAU/MAU ratio | Stickiness — how often monthly users return daily | 0.15–0.30 (casual), 0.30–0.50 (core) | Declining ratio with stable MAU = engagement erosion |
| Session length (median) | How long players stay per visit | 8–25 min (mobile), 30–90 min (PC/console) | Bimodal distribution (very short + very long) suggests onboarding issues |
| Session frequency | How often players return per week | 3–5x/week (healthy), <2x/week (at risk) | Declining frequency precedes churn by 1–2 weeks |
| D1/D7/D30 retention | Percentage returning after N days | D1: 35–50%, D7: 15–25%, D30: 5–12% | D1 < 30% = critical onboarding problem |

### Progression Metrics

| Metric | What It Measures | Healthy Range | Warning Signal |
|---|---|---|---|
| Time-to-milestone | How long to reach key progression points | Varies by design | Large variance = inconsistent difficulty |
| Completion rate by content | What percentage finish each piece of content | 60–85% for main path | Below 50% = content is too hard or unengaging |
| Funnel drop-off | Where players stop progressing | Gradual taper is expected | Sharp cliff at a single point = specific blocker |
| Skill distribution | Bell curve of player performance | Normal distribution | Heavy left skew = too hard; heavy right skew = too easy |

### Economy Metrics

| Metric | What It Measures | Healthy Range | Warning Signal |
|---|---|---|---|
| Currency velocity | How fast currency circulates (earned → spent) | Spend within 1–3 sessions of earning | Hoarding (>5 sessions) = nothing worth buying |
| Inflation index | Currency supply growth vs. sink absorption | Net supply growth < 5%/week | Accelerating growth = sinks are failing |
| Wealth Gini coefficient | Inequality of currency distribution | 0.3–0.5 | >0.7 = extreme inequality, economy is stratified |
| Sink engagement | Percentage of players using each currency sink | Top sinks > 40% participation | No sink above 20% = economy has no compelling drains |

### Balance Metrics

| Metric | What It Measures | Healthy Range | Warning Signal |
|---|---|---|---|
| Pick rate | How often each option is selected | Roughly uniform ± genre expectations | Any option below 2% or above 40% in a pool of 10+ |
| Win rate | How often each option wins when picked | 45–55% in symmetric games | Consistent >55% across skill levels = overpowered |
| Match duration | How long competitive matches last | Within ±30% of target duration | High variance = snowball/stall problem |
| Build diversity | Number of viable builds/strategies | Increases with player skill | If diversity decreases with skill = false choice problem |

### UX Metrics

| Metric | What It Measures | Healthy Range | Warning Signal |
|---|---|---|---|
| Time-to-first-action | How long before the player does something meaningful | < 30 seconds | > 60 seconds = too many gates before play |
| Tutorial completion | Percentage finishing the tutorial flow | > 80% | < 60% = tutorial is too long or unclear |
| Help/hint usage | How often players seek assistance | Decreasing over sessions | Increasing or flat = systems aren't