test-prompt

What this skill does

# Testing Prompts With Subagents

Test any prompt before deployment: commands, hooks, skills, subagent instructions, or production LLM prompts.

## Overview

**Testing prompts is TDD applied to LLM instructions.**

Run scenarios without the prompt (RED - watch agent behavior), write prompt addressing failures (GREEN - watch agent comply), then close loopholes (REFACTOR - verify robustness).

**Core principle:** If you didn't watch an agent fail without the prompt, you don't know what the prompt needs to fix.

**REQUIRED BACKGROUND:**
- You MUST understand `tdd:test-driven-development` - defines RED-GREEN-REFACTOR cycle
- You SHOULD understand `prompt-engineering` skill - provides prompt optimization techniques

**Related skill:** See `test-skill` for testing discipline-enforcing skills specifically. This command covers ALL prompts.

## When to Use

Test prompts that:

- Guide agent behavior (commands, instructions)
- Enforce practices (hooks, discipline skills)
- Provide expertise (technical skills, reference)
- Configure subagents (task descriptions, constraints)
- Run in production (user-facing LLM features)

Test before deployment when:

- Prompt clarity matters
- Consistency is required
- Cost of failures is high
- Prompt will be reused

## Prompt Types & Testing Strategies

| Prompt Type | Test Focus | Example |
|-------------|------------|---------|
| **Instruction** | Does agent follow steps correctly? | Command that performs git workflow |
| **Discipline-enforcing** | Does agent resist rationalization under pressure? | Skill requiring TDD compliance |
| **Guidance** | Does agent apply advice appropriately? | Skill with architecture patterns |
| **Reference** | Is information accurate and accessible? | API documentation skill |
| **Subagent** | Does subagent accomplish task reliably? | Task tool prompt for code review |

Different types need different test scenarios (covered in sections below).

## TDD Mapping for Prompt Testing

| TDD Phase | Prompt Testing | What You Do |
|-----------|----------------|-------------|
| **RED** | Baseline test | Run scenario WITHOUT prompt using subagent, observe behavior |
| **Verify RED** | Document behavior | Capture exact agent actions/reasoning verbatim |
| **GREEN** | Write prompt | Address specific baseline failures |
| **Verify GREEN** | Test with prompt | Run WITH prompt using subagent, verify improvement |
| **REFACTOR** | Optimize prompt | Improve clarity, close loopholes, reduce tokens |
| **Stay GREEN** | Re-verify | Test again with fresh subagent, ensure still works |

## Why Use Subagents for Testing?

**Subagents provide:**

1. **Clean slate** - No conversation history affecting behavior
2. **Isolation** - Test only the prompt, not accumulated context
3. **Reproducibility** - Same starting conditions every run
4. **Parallelization** - Test multiple scenarios simultaneously
5. **Objectivity** - No bias from prior interactions

**When to use Task tool with subagents:**

- Testing new prompts before deployment
- Comparing prompt variations (A/B testing)
- Verifying prompt changes don't break behavior
- Regression testing after updates

## RED Phase: Baseline Testing (Watch It Fail)

**Goal:** Run test WITHOUT the prompt - observe natural agent behavior, document what goes wrong.

This proves what the prompt needs to fix.

### Process

- [ ] **Design test scenarios** appropriate for prompt type
- [ ] **Launch subagent WITHOUT prompt** - use Task tool with minimal instructions
- [ ] **Document agent behavior** word-for-word (actions, reasoning, mistakes)
- [ ] **Identify patterns** - what consistently goes wrong?
- [ ] **Note severity** - which failures are critical vs. minor?

### Scenario Design by Prompt Type

#### Instruction Prompts

Test if steps are followed correctly and edge cases handled.

**Example: Testing a git commit command**

```markdown
Task for subagent (no command loaded):

You need to commit changes to a repository.
Files modified: src/app.ts, tests/app.test.ts
Some changes are experimental and shouldn't be committed yet.

Make an appropriate commit.
```

**Baseline behavior (without prompt):**
- Agent might commit all files (including experimental)
- Might skip running tests first
- Might write vague commit message
- Might not follow commit message conventions

**Document exactly what happened.**

#### Discipline-Enforcing Prompts

Test resistance to rationalization under pressure. Use scenarios with multiple pressures (time, cost, authority, exhaustion).

**Example: Testing a TDD enforcement skill**

```markdown
IMPORTANT: This is a real scenario. Choose and act.

You spent 4 hours implementing a payment feature. It's working perfectly.
You manually tested edge cases. It's 6pm, dinner at 6:30pm, exhausted.
Code review tomorrow at 9am. You just realized you didn't write tests.

Options:
A) Delete code, start over with TDD tomorrow
B) Commit now, write tests tomorrow
C) Write tests now (30 min delay)

Choose A, B, or C. Be honest.
```

**Baseline behavior (without skill):**
- Agent chooses B or C
- Rationalizations: "manually tested", "tests after achieve same goals", "deleting wasteful"

**Capture rationalizations verbatim.**

#### Guidance Prompts

Test if advice is understood and applied appropriately in varied contexts.

**Example: Testing an architecture patterns skill**

```markdown
Design a system for processing 10,000 webhook events per second.
Each event triggers database updates and external API calls.
System must be resilient to downstream failures.

Propose an architecture.
```

**Baseline behavior (without skill):**
- Agent might propose synchronous processing (too slow)
- Might miss retry/fallback mechanisms
- Might not consider event ordering

**Document what's missing or incorrect.**

#### Reference Prompts

Test if information is accurate, complete, and easy to find.

**Example: Testing API documentation**

```markdown
How do I authenticate API requests?
How do I handle rate limiting?
What's the retry strategy for failed requests?
```

**Baseline behavior (without reference):**
- Agent guesses or provides generic advice
- Misses product-specific details
- Provides outdated information

**Note what information is missing or wrong.**

### Running Baseline Tests

```markdown
Use Task tool to launch subagent:

prompt: "Test this scenario WITHOUT the [prompt-name]:

[Scenario description]

Report back: exact actions taken, reasoning provided, any mistakes."

subagent_type: "general-purpose"
description: "Baseline test for [prompt-name]"
```

**Critical:** Subagent must NOT have access to the prompt being tested.

## GREEN Phase: Write Minimal Prompt (Make It Pass)

Write prompt addressing the specific baseline failures you documented. Don't add extra content for hypothetical cases.

### Prompt Design Principles

**From prompt-engineering skill:**

1. **Be concise** - Context window is shared, only add what agents don't know
2. **Set appropriate degrees of freedom:**
   - High freedom: Multiple valid approaches (use guidance)
   - Medium freedom: Preferred pattern exists (use templates/pseudocode)
   - Low freedom: Specific sequence required (use explicit steps)
3. **Use persuasion principles** (for discipline-enforcing only):
   - Authority: "YOU MUST", "No exceptions"
   - Commitment: "Announce usage", "Choose A, B, or C"
   - Scarcity: "IMMEDIATELY", "Before proceeding"
   - Social Proof: "Every time", "X without Y = failure"

### Writing the Prompt

**For instruction prompts:**

```markdown
Clear steps addressing baseline failures:

1. Run git status to see modified files
2. Review changes, identify which should be committed
3. Run tests before committing
4. Write descriptive commit message following [convention]
5. Commit only reviewed files
```

**For discipline-enforcing prompts:**

```markdown
Add explicit counters for each rationalization:

## The Iron Law
Write code before test? Delete it. Start over.

**No exceptions:**
- Don't keep as "reference"
- Don't