rethink-surveys

What this skill does

# Rethink Surveys

A survey is a conversation with a busy person. Design it like one.

This skill packages a framework for designing, critiquing, and operationalizing surveys that capture real intent — not satisficed checkbox approval. Built from the experience of designing the Proxymate muShanghai survey (which surfaced bugs in ~6 hours of testing that a category-checkbox flow would have hidden for weeks).

## Tool affordances (read first)

The bundled `rethink-survey-mcp-server` exposes the skill's deterministic operations as MCP tools, its canonical content as MCP resources, and three framing prompts as slash-prompts. **When available, prefer them over reasoning from memory.**

- **Use MCP tools** when the user wants a *specific deterministic result*: `critique_survey` for lint, `get_template` for a use-case starter, `design_survey_session` for the staged wizard, `score_response` for a single response, `cluster_responses` for a batch.
- **Reason from skill content** when the user wants to *understand*: debate a principle, discuss trade-offs, explain why hypotheticals fail, or do anything not exposed as a tool (e.g. `/turn-into-app` codegen).
- **Fetch resources for grounding** instead of paraphrasing: `rethink://principles`, `rethink://principles/{id}`, `rethink://structure/4-part`, `rethink://question-library/{part}`, `rethink://use-case/{event|founder|gig}`, `rethink://scoring/rubrics`, `rethink://modality/decision-tree`, `rethink://anti-patterns`.
- **Slash-prompts** the user may invoke: `/jarrett-review` (Caroline Jarrett-style review), `/design-coaching` (Socratic design questions), `/mom-test-check` (Rob Fitzpatrick founder-discovery lint).

If the MCP server isn't connected, all of the below still works from this skill's reference files. For details on tool inputs/outputs, fallback rules, and prompt framing, read `references/mcp-integration.md`.

## Three operating modes (commands)

1. `/design-survey` — interactive design session: research goal → audience → hypotheses → questions → modality → output schema. Walks the user through 5–8 decisions, each with explicit trade-offs. Reuses templates where appropriate.
2. `/critique-survey` — paste-in review: identify leading questions, double-barreled items, satisficing bait, missing behavioral anchors, demographic-for-demographic noise. Returns a Jarrett-style critique with concrete rewrites.
3. `/turn-into-app` — implementation handoff: takes a finalized question set and produces app scaffold (TanStack Start / Next.js / pure HTML, plus Supabase or simple JSON storage). User picks the stack; this command emits the code.

If the user's intent is unclear, ask: *"Are you (a) starting fresh, (b) reviewing something you already wrote, or (c) ready to ship — meaning: turn an existing question set into an app?"*

## The seven design principles (the spine)

Every survey-design decision routes back to one of these. Memorize.

1. **Ask about real past behavior, not hypotheticals.** "Last time X happened, what did you do?" beats "What would you do if X happened?" by an order of magnitude in signal quality. Hypotheticals invite imagined-self answers.
2. **Don't pre-signal the right answer.** "Most people find X useful — would you?" is broken. Replace with neutral framing.
3. **Start broad and open-ended before narrowing.** Open-ended question first → respondent's authentic frame; structured questions after → that frame doesn't get lost in a checkbox.
4. **Separate screening / diagnosis / segmentation.** Screening = "are you the right respondent?" Diagnosis = "what's actually happening?" Segmentation = "what cohort do you fit into?" Mixing them leads to confusing branching and worse data.
5. **No double-barreled questions.** "Was the food fast and cheap?" should be two questions. If you can't decompose without losing the meaning, reword.
6. **Behavioral anchors over Likert scales** when anchors exist. "I'd lose ~30 minutes" is universal; "Somewhat annoying" is whatever the respondent thinks today.
7. **Honest length claim on the landing page.** If you say 60 seconds, deliver 60 seconds. Lying about length is the single fastest way to nuke completion rate. Jarrett's rule: never lie about length.

For the *why* behind each principle (Tourangeau's 4-stage cognitive model, satisficing theory, etc.) read `references/design-principles.md`.

## The 4-part hybrid structure

This is the structural skeleton every well-designed survey shares. Use it as a checklist:

| Part | Purpose | Example questions | Failure mode |
|------|---------|-------------------|--------------|
| **1. Discovery (open-ended)** | Capture authentic frame before constraining it | "What's the one thing you're worried about?" / "Last time X, what did you do?" | Skipping → checkbox-only data with zero novelty |
| **2. Diagnostic (targeted, structured)** | Make responses analyzable across users | Severity 1–5 with anchors / Frequency / Workaround text | Skipping → can't cluster or rank |
| **3. Intent & prioritization** | Surface willingness, urgency, willingness-to-pay | Top-3 priorities / WTP buckets / "When would help most?" | Skipping → no demand signal, can't prioritize |
| **4. Segmentation & follow-up** | Cohort the respondent + qualify for follow-up | Who-are-you / Group composition / Contact + split consent | Skipping → can't compare cohorts, no interview pipeline |

For each question, **tag it** with which part it serves. Questions that don't clearly belong to one part are usually candidates for cutting.

For battle-tested question patterns by part (with EN+ZH parallel text and trade-off notes), read `references/question-library.md`.

## Modality decision tree

Match modality to context:

- **Pure form (tap-to-select)** — physical QR scan, on-venue, low-attention. Target ≤60s.
- **Form with optional voice on key questions** — moderate attention; the diagnostic question benefits from authentic phrasing. Target ≤90s.
- **AI-conversational interviewer** — pre-event email warm-list, deep research mode, willing respondents. AI probes with follow-ups based on prior answers. Target 3–5 minutes, ~10 substantive answers.
- **Voice-note only** — accessibility-first / language-flexible / low-literacy contexts. Single open prompt, transcribed server-side, post-processed for cluster signals.

Default rule: **start with form + optional voice on the discovery question.** Add AI-interviewer only if the research goal requires probing depth that a flat form can't deliver.

For the full UX/UI guide (with React/HTML scaffolds and storage schema recommendations), read `references/multimodal-ux.md`.

## What to ask AI conversationally vs. show as form fields

| Show as form field | Ask via AI |
|---|---|
| Multiple-choice with finite known options | Open-ended diagnosis with potential follow-up |
| Numeric or scale (with behavioral anchors) | Causal explanation: "Why did that happen?" |
| Demographic/segmentation | Workaround: "What did you try first?" |
| Consent checkboxes | Edge cases the form options don't cover |

AI is for the parts where **you don't know what answers to expect**. Form is for the parts where you do.

## Scoring & tagging framework

After collection, every response should map to:

- **`pain_unit`** — the specific worry/problem captured in the discovery question, ideally with a cluster ID assigned by post-hoc LLM enrichment
- **`severity`** (1–5 with behavioral anchors)
- **`willingness_to_pay`** (bucket or numeric range, currency-localized for bilingual audiences)
- **`workaround_strength`** — derived from whether the respondent has an existing fallback (text length + sentiment)
- **`segment`** — derived from screening/segmentation answers
- **`interview_score`** — composite for ranking interview-call candidates: `severity × specificity × wtp_weight × consent_research_call × novelty_bonus`

For the full intent-scoring math (composite formula, tag taxonomy, cluster assignment heuristics), read `references/scoring-framework.md`.

## Use-ca