ai-feature-prd

What this skill does

# AI Feature PRD Expert

## Overview

AI and ML features break the assumptions a standard PRD takes for granted. Outputs are non-deterministic. Quality is statistical, not categorical. The "spec" is half product, half eval suite. A regular PRD that says "Search returns the top result" is replaced by "the assistant returns a helpful, harmless, on-policy answer with a refusal rate under 4% on the golden set, p95 latency under 1.8s, and cost-per-conversation under $0.05."

This skill produces an **AI Feature PRD** that extends the standard 8-section PRD (see `create-prd/`) with three additional sections built for the realities of shipping AI: **AI System Design** (Section 9), **Eval & Safety Plan** (Section 10), and **Operations & Cost** (Section 11). It draws on Andrej Karpathy's "Software 2.0" framing (the model *is* the spec), Anthropic's Responsible Scaling Policy patterns, OpenAI's GPT model-spec style for defining intended behavior, the Reforge AI PM curriculum, and the EU AI Act's risk-tier model that the team has to declare upfront.

This is a template-based skill -- no Python tool. The artifact is a markdown PRD. Pair this with `engineering/llm-cost-optimizer/` for the cost-model math and with `ra-qm-team/eu-ai-act-specialist/` for the regulatory classification.

### When to Use

- **Adding an AI/ML feature to an existing product** -- a search assistant, a recommendation, an auto-summarizer, a copilot, an agent.
- **Building an AI-first product** -- the entire surface area is model-mediated.
- **Migrating a deterministic feature to an LLM** -- replacing a rules-based system or scripted flow with a model.
- **Fine-tuning, prompt-tuning, or RAG decision** -- the PRD captures the architecture rationale so engineering does not relitigate it mid-build.
- **Regulated context** -- EU AI Act, HIPAA, FINRA, FDA SaMD -- the PRD must enumerate the risk tier, the eval bar, and the audit trail before kickoff.

### When NOT to Use

- For a non-AI feature (use `create-prd/` -- the standard 8-section PRD is sufficient).
- For pure model R&D with no product surface (use a research design doc).
- For a one-off internal prompt or batch script that does not ship to users (a Notion page is fine).
- When the AI feature has no production traffic plan -- this PRD assumes you will ship to real users.

## Why a separate AI PRD

A regular PRD assumes a deterministic function: input -> code -> output. AI flips this:

| Standard feature | AI feature |
|---|---|
| Output is the same for the same input | Output varies; same input may produce different responses |
| Quality is binary (correct / incorrect) | Quality is statistical (95% acceptance, 2% hallucination) |
| Tests verify behavior | Evals score quality on golden sets |
| Bugs are fixed in code | Bugs are reduced via prompts, fine-tunes, retrieval, or guardrails |
| Cost is roughly linear with traffic | Cost is per-token, per-call, per-context-window |
| Rollback is a code revert | Rollback may require a different model + prompt + retrieval combo |
| Failure modes are predictable | Failure modes include jailbreaks, hallucination, drift, prompt injection |

The standard PRD has no place to declare a hallucination budget, an eval golden set, a model-fallback strategy, or a refusal policy. This skill adds those sections.

## AI PRD Framework (11 Sections)

Sections 1-8 follow the standard PRD from `create-prd/` (Summary, Contacts, Background, Objective, Market Segments, Value Proposition, Solution, Release). The summary below is abbreviated; full guidance lives in `create-prd/SKILL.md`. Sections 9-11 are AI-specific and are the heart of this skill.

### Section 1: Summary
Two to three sentences. What the AI feature is, who it is for, why it matters now. State the model class plainly ("a Claude Sonnet-4.5 based assistant", "a fine-tuned BERT classifier") -- abstraction here hides cost and risk.

### Section 2: Contacts
PM, Eng Lead, Design Lead, ML/Applied Research Lead, Safety/Trust reviewer, Legal/Compliance reviewer. AI features need more cross-functional sign-off than standard features.

### Section 3: Background
Why now? What changed? Frontier models capable of this task, new internal data, regulatory clarity, customer demand, competitor shipping. Cite the model capability or research that unblocks this -- it dates the PRD honestly.

### Section 4: Objective
Business benefit + customer benefit + 2-4 SMART Key Results. AI KRs should include at least one quality metric (acceptance, win-rate, hallucination) **and** one cost metric.

### Section 5: Market Segments
Jobs-to-be-done, not demographics. Be explicit about which segments are excluded for safety reasons (e.g., minors, healthcare, legal advice contexts).

### Section 6: Value Proposition
What jobs, gains, pains for each segment? Be explicit about the "AI premium" -- what does an AI-mediated experience offer that the deterministic baseline cannot?

### Section 7: Solution
Standard UX, key features (P0/P1/P2), assumptions. Add a **prompt sketch** -- the rough system prompt + tool definitions you intend to use. The full prompt lives in source control, but the PRD shows the contract.

### Section 8: Release
T-shirt sizing, v1 scope, deferred items, success criteria. AI features almost always release on a **shadow -> internal -> canary -> percent-rollout -> GA** ramp; declare the gates between stages.

---

### Section 9: AI System Design

The architectural decisions for the AI subsystem.

**9.1 Model selection rationale**

| Field | Example |
|---|---|
| Primary model | `claude-sonnet-4.5` (Anthropic, via Bedrock) |
| Fallback model | `claude-haiku-4` for cost cap; `gpt-5-mini` for outage |
| Reason chosen | Best quality on our eval set; cheaper than `opus`; safety profile fits |
| Update policy | Re-evaluate primary every 90 days against new releases |
| Vendor risk | Single-vendor risk mitigated by `gpt-5-mini` fallback |

State a primary, a fallback, and a switch trigger. "Model lock-in" is the most expensive technical debt in AI products -- the PRD names the exit ramp.

**9.2 Architecture pattern (Prompt vs Fine-tune vs RAG vs Agent)**

| Pattern | When to choose | Cost profile | Engineering complexity |
|---|---|---|---|
| Prompt only | Behavior steerable via instructions; tasks broadly within base-model capability | Low (per-token) | Low |
| Few-shot / system prompt with examples | Stylistic consistency or format requirements | Low-medium | Low |
| RAG | Output must cite up-to-date private knowledge | Medium (vector store + LLM) | Medium |
| Fine-tune (SFT) | Domain-specific tone, format, or structured-output reliability | Medium-high (training + inference) | Medium-high |
| RLHF / preference fine-tune | Subjective quality the base model gets wrong | High | High |
| Agentic / tool use | Multi-step task requiring tool invocation | High (per-tool-call) | High |

Pick one (or a combination) and **state the rejected alternatives with reasons**. The Reforge AI PM curriculum is explicit: 80% of "we need a fine-tune" decisions are actually "we need better prompts and retrieval first."

**9.3 Data flow**

Diagram or describe: input -> preprocessing -> (retrieval?) -> model -> postprocessing -> output. Explicitly mark:

- What user data leaves the boundary (and to which vendor, under which DPA)
- PII handling (redaction, hashing, exclusion)
- Retention policy (user-input retention, model-output retention, training-opt-out)
- Logging strategy (what is logged, for how long, who can access)

**9.4 Prompt contract**

Sketch the system prompt structure. Production prompts live in source control with versioning, but the PRD captures the contract:

```
Role: <who the assistant is>
Goal: <task>
Inputs: <what context is passed>
Constraints: <refusals, scope, style>
Output format: <schema or example>
Tools: <list with one-line descriptions>
```

### Section 10: Eval & Safety Plan

This is the section that separates a real AI PRD from a thinly-rebadged one.

**10.1 Eval criteria