quiz-generator

What this skill does

# Quiz Generator for Intelligent Textbooks

**Version:** 0.4

## Overview
1. For each markdown chapter, generate interactive multiple-choice quizzes for textbook chapters with quality distractor analysis.
2. Generate quality reports in markdown format.
3. Update mkdocs.yml navigation to include quizzes and reports.

## Purpose

This skill automates quiz creation for intelligent textbooks by analyzing chapter content to generate contextually relevant multiple-choice questions. Each quiz is aligned to specific concepts from the learning graph, distributed across Bloom's Taxonomy cognitive levels, and formatted using mkdocs-material question admonition format with upper-alpha (A, B, C, D) answer choices. The skill ensures quality distractors, balanced answer distribution, and comprehensive explanations for educational value.

## When to Use This Skill

Use this skill after:

1. Chapter content has been generated or written (1000+ words per chapter)
2. Learning graph exists with concept dependencies
3. Glossary is available (recommended for terminology questions)

Trigger this skill when:

- Creating quizzes for new chapters
- Updating quizzes after content revisions
- Building comprehensive quiz bank for entire textbook
- Exporting quiz data for LMS or chatbot integration

## Token Efficiency: Serial Execution Only

!!! warning "NEVER Use Parallel Agents Unless the User Explicitly Requests It"
    **Always use a single serial agent for quiz generation.** This is a hard
    requirement, not a suggestion. Do not offer parallel execution as an option.

Each spawned agent incurs ~12,000 tokens of startup overhead (system prompt,
tool schemas, project context). Parallel execution multiplies this overhead
with zero quality benefit. Measured data from a 17-chapter quiz generation:

| Approach | System Overhead | Total Tokens | Waste |
|----------|----------------|--------------|-------|
| **Serial (1 agent)** | ~12,000 | ~310,000 | — |
| Parallel (4 agents) | ~48,000 | ~358,000 | +48,000 (13%) |

Additional problems with parallel execution observed in production:

1. **Inconsistent behavior:** Agents independently choose different strategies,
   causing 5x variation in tool calls (8 vs 39) for the same workload
2. **Conflicting file edits:** Multiple agents modifying mkdocs.yml simultaneously
   causes inconsistencies requiring manual repair
3. **No shared learning:** Each agent starts from scratch with no accumulated
   context from earlier chapters

The skill supports two modes:
- **Serial mode** (default, always): One agent processes all chapters sequentially
- **Single chapter mode**: Generate quiz for one specific chapter

## Workflow

### Phase 1: Setup (Sequential)

This phase runs once before quiz generation, reading shared context.

#### Step 1.1: Capture Start Time

```bash
date "+%Y-%m-%d %H:%M:%S"
```

Log the start time for the session report.

#### Step 1.2: Indicate Skill Running

Notify the user: "Quiz Generator Skill v0.4 running in serial mode."

#### Step 1.3: Read Shared Context

Read and cache these files for all agents:

1. **Course Description** (`docs/course-description.md`)
   - Extract target audience and reading level
   - Note Bloom's Taxonomy learning outcomes

2. **Learning Graph** (`docs/learning-graph/learning-graph.csv` or similar)
   - Load concept list with dependencies
   - Calculate concept centrality for prioritization

3. **Glossary** (`docs/glossary.md`)
   - Load term definitions for terminology questions
   - Note which concepts have glossary entries

4. **Chapter List** (scan `docs/chapters/` directory)
   - Enumerate all chapter directories
   - Count words per chapter for readiness assessment

#### Step 1.4: Assess Content Readiness

Calculate content readiness score (1-100) for each target chapter:

**Quality Checks:**

##### 1. **Chapter word count:**
   - 2000+ words = excellent (20 pts)
   - 1000-1999 words = good (15 pts)
   - 500-999 words = basic (10 pts)
   - <500 words = insufficient (5 pts)

##### 2. **Example coverage:**
   - 60%+ concepts with examples = excellent (20 pts)
   - 40-59% = good (15 pts)
   - 20-39% = basic (10 pts)
   - <20% = insufficient (5 pts)

##### 3. **Glossary coverage:**
   - 80%+ chapter concepts defined = excellent (20 pts)
   - 60-79% = good (15 pts)
   - 40-59% = basic (10 pts)
   - <40% = insufficient (5 pts)

##### 4. **Concept clarity:**
   - Clear explanations for all concepts (20 pts)
   - Most concepts clear (15 pts)
   - Some unclear concepts (10 pts)
   - Many unclear concepts (5 pts)

##### 5. **Learning graph alignment:**
   - All chapter concepts mapped (20 pts)
   - Most mapped (15 pts)
   - Some mapped (10 pts)
   - Few mapped (5 pts)

**Content Readiness Ranges:**

- 90-100: Rich content, excellent quiz quality possible
- 70-89: Good content, solid quiz possible
- 50-69: Basic content, limited quiz possible
- Below 50: Insufficient content for quality quiz

**User Dialog Triggers:**

- Score < 60: Ask "Chapter [X] has limited content ([N] words). Generate shorter quiz or skip?"
- No glossary: Ask "No glossary found. Definition questions will be limited. Proceed?"
- Concept gaps: Ask "[N] concepts in chapter not in learning graph. Continue with available concepts?"
- No learning outcomes: Ask "No Bloom's Taxonomy outcomes in course description. Use default distribution?"

### Phase 2: Quiz Generation (Serial)

Launch ONE agent that processes all chapters sequentially. The agent reads each
chapter, generates 10 questions, and writes the quiz file before moving to the
next chapter. This pays the ~12K system prompt overhead only once.

**Agent Prompt Template:**

```
You are generating quizzes for an intelligent textbook. Generate quizzes for
ALL of the following chapters, processing them one at a time.

COURSE CONTEXT:
- Course: [course name]
- Target audience: [audience]
- Reading level: [level]

BLOOM'S TAXONOMY TARGETS:
- Introductory chapters (1-3): 40% Remember, 40% Understand, 15% Apply, 5% Analyze
- Intermediate chapters (4-N): 25% Remember, 30% Understand, 30% Apply, 15% Analyze
- Advanced chapters: 15% Remember, 20% Understand, 25% Apply, 25% Analyze, 10% Evaluate, 5% Create

CHAPTERS TO PROCESS:
[List ALL chapter directories with full paths]

FOR EACH CHAPTER:
1. Read the chapter content at the index.md file
2. Identify the key concepts covered in that chapter
3. Generate exactly 10 questions following the format below
4. Ensure answer balance: A (2-3), B (2-3), C (2-3), D (2-3)
5. Write the quiz to docs/chapters/[chapter-dir]/quiz.md

QUIZ FORMAT - Each question MUST follow this exact format:

#### [N]. [Question text ending with ?]

<div class="upper-alpha" markdown>
1. [Option A text]
2. [Option B text]
3. [Option C text]
4. [Option D text]
</div>

??? question "Show Answer"
    The correct answer is **[LETTER]**. [Explanation 50-100 words]

    **Concept Tested:** [Concept Name]

---

QUIZ FILE STRUCTURE:
# Quiz: [Chapter Title]

Test your understanding of [topic] with these review questions.

---

[Questions 1-10 following the format above]

REPORT when done:
- Chapter name
- Number of questions
- Bloom's distribution (R:#, U:#, Ap:#, An:#)
- Answer distribution (A:#, B:#, C:#, D:#)
```

### Phase 2 Steps (Per Chapter)

#### Step 2: Determine Target Distribution

Based on chapter type (introductory, intermediate, advanced), set target Bloom's Taxonomy distribution:

**Introductory Chapters (typically chapters 1-3):**
- 40% Remember
- 40% Understand
- 15% Apply
- 5% Analyze
- 0% Evaluate
- 0% Create

**Intermediate Chapters:**
- 25% Remember
- 30% Understand
- 30% Apply
- 15% Analyze
- 0% Evaluate
- 0% Create

**Advanced Chapters:**
- 15% Remember
- 20% Understand
- 25% Apply
- 25% Analyze
- 10% Evaluate
- 5% Create

Determine chapter type by:
- Position in textbook (first 3 chapters = introductory)
- Concept centrality in learning graph (high centrality = advanced)
- Explicit markers in chapter metadata
- User specification

Target question count: 8