validation-frameworks

What this skill does

# Validation Frameworks

Frameworks for validating problems, solutions, and product assumptions before committing to full development.

## When to Use This Skill

**Auto-loaded by agents**:

- `research-ops` - For problem/solution validation and assumption testing

**Use when you need**:

- Validating product ideas before building
- Testing assumptions and hypotheses
- De-risking product decisions
- Running MVP experiments
- Validating problem-solution fit
- Testing willingness to pay
- Evaluating technical feasibility

## Core Principle: Reduce Risk Through Learning

Building the wrong thing is expensive. Validation reduces risk by answering critical questions before major investment:

- **Problem validation**: Is this a real problem worth solving?
- **Solution validation**: Does our solution actually solve the problem?
- **Market validation**: Will people pay for this?
- **Usability validation**: Can people use it?
- **Technical validation**: Can we build it?

## The Validation Spectrum

Validation isn't binary (validated vs. not validated). It's a spectrum of confidence:

```
Wild Guess → Hypothesis → Validated Hypothesis → High Confidence → Proven
```

Early stage: Cheap, fast tests (low confidence gain)
Later stage: More expensive tests (high confidence gain)

**Example progression**:

1. **Assumption**: "Busy parents struggle to plan healthy meals"
2. **Interview 5 parents** → Some validation (small sample)
3. **Survey 100 parents** → More validation (larger sample)
4. **Prototype test with 20 parents** → Strong validation (behavior observed)
5. **Launch MVP, track engagement** → Very strong validation (real usage)
6. **Measure retention after 3 months** → Proven (sustained behavior)

## Problem Validation vs. Solution Validation

### Problem Validation

**Question**: "Is this a problem worth solving?"

**Goal**: Confirm that:

- The problem exists
- It's painful enough that people want it solved
- It's common enough to matter
- Current solutions are inadequate

**Methods**:

- Customer interviews (discovery-focused)
- Ethnographic observation
- Surveys about pain points
- Data analysis (support tickets, analytics)
- Jobs-to-be-Done interviews

**Red flags that stop here**:

- No one cares about this problem
- Existing solutions work fine
- Problem only affects tiny niche
- Pain point is mild annoyance, not real pain

**Output**: Problem statement + evidence it's worth solving

See `assets/problem-validation-canvas.md` for a ready-to-use framework.

---

### Solution Validation

**Question**: "Does our solution solve the problem?"

**Goal**: Confirm that:

- Our solution addresses the problem
- Users understand it
- Users would use it
- It's better than alternatives
- Value proposition resonates

**Methods**:

- Prototype testing
- Landing page tests
- Concierge MVP (manual solution)
- Wizard of Oz (fake backend)
- Pre-sales or waitlist signups

**Red flags**:

- Users don't understand the solution
- They prefer their current workaround
- Would use it but not pay for it
- Solves wrong part of the problem

**Output**: Validated solution concept + evidence of demand

See `assets/solution-validation-checklist.md` for validation steps.

---

## The Assumption-Validation Cycle

### 1. Identify Assumptions

Every product idea rests on assumptions. Make them explicit.

**Types of assumptions**:

**Desirability**: Will people want this?

- "Users want to track their habits"
- "They'll pay $10/month for this"
- "They'll invite their friends"

**Feasibility**: Can we build this?

- "We can process data in under 1 second"
- "We can integrate with their existing tools"
- "Our team can build this in 3 months"

**Viability**: Should we build this?

- "Customer acquisition cost will be under $50"
- "Retention will be above 40% after 30 days"
- "We can reach 10k users in 12 months"

**Best practice**: Write assumptions as testable hypotheses

- Not: "Users need this feature"
- Yes: "At least 60% of users will use this feature weekly"

---

### 2. Prioritize Assumptions to Test

Not all assumptions are equally important. Prioritize by:

**Risk**: How uncertain are we? (Higher risk = higher priority)
**Impact**: What happens if we're wrong? (Higher impact = higher priority)

**Prioritization matrix**:

| Risk/Impact   | High Impact    | Low Impact  |
| ------------- | -------------- | ----------- |
| **High Risk** | **Test first** | Test second |
| **Low Risk**  | Test second    | Maybe skip  |

**Riskiest assumptions** (test these first):

- Leap-of-faith assumptions the entire business depends on
- Things you've never done before
- Assumptions about user behavior with no data
- Technical feasibility of core functionality

**Lower-risk assumptions** (test later or assume):

- Based on prior experience
- Easy to change if wrong
- Industry best practices
- Minor features

---

### 3. Design Validation Experiments

For each assumption, design the cheapest test that could prove it wrong.

**Experiment design principles**:

**1. Falsifiable**: Can produce evidence that assumption is wrong
**2. Specific**: Clear success/failure criteria defined upfront
**3. Minimal**: Smallest effort needed to learn
**4. Fast**: Results in days/weeks, not months
**5. Ethical**: Don't mislead or harm users

**The Experiment Canvas**: See `assets/validation-experiment-template.md`

---

### 4. Run the Experiment

**Before starting**:

- [ ] Define success criteria ("At least 40% will click")
- [ ] Set sample size ("Test with 50 users")
- [ ] Decide timeframe ("Run for 1 week")
- [ ] Identify what success/failure would mean for product

**During**:

- Track metrics rigorously
- Document unexpected learnings
- Don't change experiment mid-flight

**After**:

- Analyze results honestly (avoid confirmation bias)
- Document what you learned
- Decide: Pivot, persevere, or iterate

---

## Validation Methods by Fidelity

### Low-Fidelity (Hours to Days)

**1. Customer Interviews**

- **Cost**: Very low (just time)
- **Validates**: Problem existence, pain level, current solutions
- **Limitations**: What people say ≠ what they do
- **Best for**: Early problem validation

**2. Surveys**

- **Cost**: Low
- **Validates**: Problem prevalence, feature preferences
- **Limitations**: Response bias, can't probe deeply
- **Best for**: Quantifying what you learned qualitatively

**3. Landing Page Test**

- **Cost**: Low (1-2 days to build)
- **Validates**: Interest in solution, value proposition clarity
- **Measure**: Email signups, clicks to "Get Started"
- **Best for**: Demand validation before building

**4. Paper Prototypes**

- **Cost**: Very low (sketch on paper/whiteboard)
- **Validates**: Concept understanding, workflow feasibility
- **Limitations**: Low realism
- **Best for**: Very early solution concepts

---

### Medium-Fidelity (Days to Weeks)

**5. Clickable Prototypes**

- **Cost**: Medium (design tool, 2-5 days)
- **Validates**: User flow, interaction patterns, comprehension
- **Tools**: Figma, Sketch, Adobe XD
- **Best for**: Usability validation pre-development

**6. Concierge MVP**

- **Cost**: Medium (your time delivering manually)
- **Validates**: Value of outcome, willingness to use/pay
- **Example**: Manually curate recommendations before building algorithm
- **Best for**: Validating value before automation

**7. Wizard of Oz MVP**

- **Cost**: Medium (build facade, manual backend)
- **Validates**: User behavior, feature usage, workflows
- **Example**: "AI" feature that's actually humans behind the scenes
- **Best for**: Testing expensive-to-build features

---

### High-Fidelity (Weeks to Months)

**8. Functional Prototype**

- **Cost**: High (weeks of development)
- **Validates**: Technical feasibility, actual usage patterns
- **Limitations**: Expensive if you're wrong
- **Best for**: After other validation, final pre-launch check

**9. Private Beta**

- **Cost**: High (full build + support)
- **Validates**: Real-world usage, retention, bugs
- **Best for**: Pre-launch final vali