assessment-builder

What this skill does

## Purpose

The assessment-builder skill helps educators create comprehensive, balanced assessments that measure conceptual understanding (not just memorization). This skill generates varied question types, designs meaningful distractors for MCQs, aligns questions with Bloom's taxonomy levels, and provides rubrics for open-ended questions.

**Constitution v4.0.1 Alignment**: This skill implements evals-first assessment design (foundational to all pillars)—defining success criteria BEFORE creating assessments, and integrating Section IIb (AI Three Roles Framework) co-learning evaluation.

## Evals-First Assessment Design (Constitution v4.0.1)

**CRITICAL WORKFLOW**:
1. **Evals First**: Define success criteria from chapter spec BEFORE designing questions
2. **Objectives Second**: Map questions to learning objectives (from spec)
3. **Questions Third**: Design assessment items that test evals
4. **Validation Fourth**: Verify questions actually measure what evals define

**Template for Assessment Planning**:
```markdown
### Assessment Planning (Evals-First)

**Source**: Chapter spec at `specs/part-X/chapter-Y/spec.md`

**Success Evals from Spec**:
1. 75%+ students write valid specification (measured by exercise)
2. 80%+ identify vague requirements (measured by quiz)
3. Students demonstrate co-learning (measured by reflection)

**Learning Objectives** (from spec):
- LO-001: Write clear specifications
- LO-002: Identify ambiguous requirements
- LO-003: Work effectively with AI partner

**Assessment Design**:
- Q1-3: Test LO-001 (spec writing) → Eval #1
- Q4-5: Test LO-002 (identify vagueness) → Eval #2
- Q6: Test LO-003 (co-learning) → Eval #3
```

**Do NOT** create assessments without:
- ✅ Reference to approved spec with evals
- ✅ Explicit mapping: Question → Objective → Eval
- ✅ Validation that questions measure defined success criteria

## When to Activate

Use this skill when:
- Educators need to create quizzes, tests, or exams
- Designing assessments aligned with learning objectives
- Creating varied question types (MCQ, code-writing, debugging, projects)
- Need meaningful distractors based on common student errors
- Want balanced cognitive distribution (not just recall questions)
- Generating rubrics for open-ended programming questions
- Evaluating existing assessments for quality and balance
- Ensuring 60%+ questions test application or higher-order thinking

## Inputs

Required:
- **Learning objectives**: What learners should be able to demonstrate
- **Concept/topic**: Python concepts to assess

Optional:
- **Number of questions**: How many questions to generate
- **Question types**: Preferred formats (MCQ, code-writing, debugging, etc.)
- **Target audience**: beginner | intermediate | advanced
- **Time limit**: Available assessment time
- **Bloom's distribution**: Desired cognitive level mix

## Process

### Step 1: Clarify Assessment Goals

Understand:
- What specific skills/knowledge to assess
- Depth of understanding required
- Format constraints (time, delivery method)
- Stakes (formative check vs. high-stakes exam)

### Step 2: Load Question Type Reference

Read question type patterns:

```bash
Read reference/question-types.md
```

Available types:
- **MCQ**: Concept recognition, decision-making
- **Code-completion**: Apply syntax, fill strategic gaps
- **Code-tracing**: Test mental execution model
- **Debugging**: Error recognition and fixing
- **Code-writing**: Implementation from specification
- **Explanation**: Test conceptual understanding
- **Code-review**: Evaluate code quality
- **Prediction**: Anticipate behavior
- **Comparison**: Distinguish approaches
- **Project**: Integrate multiple concepts

### Step 3: Load Bloom's Assessment Alignment

Read cognitive level guidelines:

```bash
Read reference/blooms-assessment-alignment.md
```

Map question types to Bloom's levels:
- **Remember** (10-15%): MCQ recall, terminology
- **Understand** (15-20%): Tracing, prediction, explanation
- **Apply** (30-40%): Code-completion, simple code-writing
- **Analyze** (20-25%): Debugging, comparison, code-review
- **Evaluate** (10-15%): Best-approach selection, trade-offs
- **Create** (5-10%): Projects, design problems

**Target**: 60%+ non-recall (Apply and higher)

### CEFR Proficiency Integration (Constitution v3.1.2)

Map assessment difficulty to CEFR proficiency levels (aligned with skills-proficiency-mapper):

**A1 (Beginner - Recognition)**:
- MCQs: Recognize syntax, identify basic concepts
- Code-tracing: Predict simple, linear code output
- No debugging or design questions yet

**A2 (Elementary - Guided Application)**:
- MCQs: Choose correct approach with scaffolding
- Code-completion: Fill strategic blanks with hints
- Simple debugging: Identify syntax errors with guidance

**B1 (Intermediate - Independent Application)**:
- Code-writing: Implement from clear specification
- Debugging: Find and fix logic errors independently
- Code-review: Evaluate simple code quality

**B2 (Upper-Intermediate - Analysis)**:
- Design questions: Choose appropriate data structures/algorithms
- Optimization: Identify performance improvements
- Trade-off evaluation: Compare multiple valid approaches

**C1 (Advanced - Synthesis)**:
- Architecture design: Plan system structure
- Complex debugging: Diagnose multi-layered issues
- Best practices justification: Defend design decisions

**Assessment Design Rule**: Questions must match lesson's target CEFR level (from spec).

### Three-Role AI Partnership Assessment (Section IIb, Constitution v4.0.1)

**CRITICAL**: AI-native development requires assessing students' ability to work WITH AI in bidirectional co-learning partnership (per Section IIb forcing functions), not just independently.

**AI's Three Roles - Assessment Types**:

**1. AI as Teacher (Does student learn from AI?)**
- Reflection question: "What did AI suggest that you hadn't considered?"
- Evaluation: "Which of AI's three suggestions was most valuable? Why?"
- Transfer: "How would you apply AI's pattern to a new problem?"

**2. AI as Student (Does student effectively teach AI?)**
- Specification quality: "Write a spec that produces correct code on first try"
- Feedback quality: "Improve AI's output by providing clear corrections"
- Iteration: "Refine your prompt based on AI's initial response"

**3. AI as Co-Worker (Does student collaborate effectively?)**
- Convergence: "Show iteration where you and AI refined solution together"
- Decision-making: "Which strategic decisions did you make vs. AI's tactical suggestions?"
- Validation: "How did you verify AI's output was correct?"

**Example Assessment Items**:

```markdown
### Q: Co-Learning Reflection (10 points)

In the previous exercise, you worked with AI to implement authentication.

1. What pattern or approach did AI suggest that you hadn't considered? (5 pts)
2. How did you validate that AI's suggestion was appropriate? (5 pts)

**Rubric**:
- Excellent (10): Specific AI suggestion identified, clear validation method
- Good (7): General AI contribution mentioned, basic validation
- Fair (4): Vague answer, no validation mentioned
- Poor (0): No evidence of learning from AI
```

**Assessment Balance for AI-Native Content**:
- 60-70%: Traditional skills (code-writing, debugging)
- 20-30%: Co-learning skills (working WITH AI)
- 10-20%: Validation/verification skills (checking AI outputs)

### Step 4: Design Varied Question Set

Create 5-10 questions with variety:

**Example Distribution**:
```
Q1: MCQ (Understand) - Concept check
Q2: Code-tracing (Understand) - Predict output
Q3: Code-completion (Apply) - Fill strategic blank
Q4: Code-writing (Apply) - Implement function
Q5: Debugging (Analyze) - Find and fix errors
Q6: Code-review (Evaluate) - Assess quality
Q7: Project (Create) - Integrate concepts
```