performance-tracking

What this skill does

# Performance Tracking

**Version:** 1.0.0
**Purpose:** Track agent, skill, and model performance metrics for continuous optimization
**Status:** Production Ready

## Overview

Performance tracking transforms workflows from "fire and forget" to **data-driven optimization systems**. By measuring what actually works, you can route tasks more effectively, identify failing patterns early, and reduce costs.

This skill provides battle-tested patterns for:
- **Agent success tracking** (completion rates, confidence scores, task type affinity)
- **Skill effectiveness** (activation counts, success correlation, usage patterns)
- **Model performance** (latency, cost, quality, provider comparison)
- **Routing optimization** (tier distribution, routing accuracy, cost efficiency)
- **Historical analysis** (trend detection, degradation alerts, pattern discovery)

Performance tracking enables **continuous improvement** by providing the data needed to make informed decisions about agent selection, model choice, and workflow routing.

### Why Track Performance

**Optimize Routing:**
- Identify which agents excel at specific task types
- Route complex tasks to high-confidence agents
- Avoid agents with low success rates for critical work

**Identify Failing Agents:**
- Detect agents with <70% success rate
- Alert when agent performance degrades
- Replace or retrain underperforming agents

**Reduce Costs:**
- Find cost-effective model alternatives
- Identify expensive agents with low success rates
- Optimize tier thresholds based on actual performance

**Improve Quality:**
- Track correlation between confidence scores and success
- Identify patterns in successful implementations
- Learn which models produce best results for task types

### What We Track

**Agent Metrics:**
- Total runs, success/failure counts
- Average confidence scores
- Task type distribution
- Last used timestamp
- Individual execution history

**Skill Metrics:**
- Activation counts per skill
- Last activation timestamps
- Success correlation (when skill active, what's success rate?)
- Co-activation patterns

**Model Metrics:**
- Total runs, success/failure counts
- Average latency (response time)
- Total cost (cumulative spend)
- Cost per successful task
- Last used timestamp

**Routing Metrics:**
- Tier distribution (how often each tier selected)
- Routing accuracy (did tier match complexity?)
- Cost efficiency (tier1 vs tier4 cost ratio)
- Decision history with outcomes

### Integration with task-complexity-router

The performance tracker provides critical feedback to the task-complexity-router:

```
Routing Feedback Loop:

1. Router selects tier based on complexity
   → task-complexity-router analyzes task
   → Routes to tier2 (medium complexity)

2. Agent executes task
   → Records: tier=2, agent=ui-developer, result=success

3. Performance tracker updates metrics
   → tier2 usage +1
   → ui-developer success +1
   → Confidence in tier2 routing increases

4. Future routing decisions informed by history
   → Router sees tier2 has 85% success rate
   → Router sees ui-developer excels at UI tasks
   → Router confidently routes similar tasks to tier2
```

---

## Metrics Schema

### JSON Structure (Version 1.0.0)

Store performance metrics in `.claude/agent-performance.json`:

```json
{
  "schemaVersion": "1.0.0",
  "lastUpdated": "2026-01-28T15:30:00Z",
  "agents": {
    "ui-developer": {
      "totalRuns": 42,
      "successCount": 38,
      "failureCount": 4,
      "avgConfidence": 0.85,
      "lastUsed": "2026-01-28T15:30:00Z",
      "taskTypes": {
        "implement-component": 15,
        "fix-styling": 12,
        "refactor-ui": 8,
        "review-code": 7
      },
      "history": [
        {
          "timestamp": "2026-01-28T15:30:00Z",
          "taskType": "implement-component",
          "result": "success",
          "confidence": 0.90,
          "duration": 45000,
          "tier": 2,
          "model": "claude-sonnet-4-5-20250929"
        },
        {
          "timestamp": "2026-01-28T14:20:00Z",
          "taskType": "fix-styling",
          "result": "success",
          "confidence": 0.85,
          "duration": 30000,
          "tier": 1,
          "model": "claude-sonnet-4-5-20250929"
        }
      ]
    },
    "backend-developer": {
      "totalRuns": 28,
      "successCount": 25,
      "failureCount": 3,
      "avgConfidence": 0.88,
      "lastUsed": "2026-01-28T14:00:00Z",
      "taskTypes": {
        "implement-api": 12,
        "fix-bug": 8,
        "database-migration": 5,
        "write-tests": 3
      },
      "history": []
    }
  },
  "skills": {
    "multi-model-validation": {
      "activations": 15,
      "lastActivated": "2026-01-28T15:00:00Z",
      "successCorrelation": 0.92,
      "coActivations": {
        "quality-gates": 12,
        "error-recovery": 8
      }
    },
    "task-complexity-router": {
      "activations": 68,
      "lastActivated": "2026-01-28T15:30:00Z",
      "successCorrelation": 0.85,
      "coActivations": {
        "multi-agent-coordination": 45,
        "hierarchical-coordinator": 30
      }
    }
  },
  "models": {
    "claude-sonnet-4-5-20250929": {
      "totalRuns": 120,
      "successCount": 108,
      "failureCount": 12,
      "avgLatency": 2500,
      "totalCost": 0.45,
      "lastUsed": "2026-01-28T15:30:00Z",
      "taskTypePerformance": {
        "code-review": { "success": 25, "failure": 2 },
        "implementation": { "success": 40, "failure": 5 },
        "testing": { "success": 20, "failure": 3 }
      }
    },
    "x-ai/grok-code-fast-1": {
      "totalRuns": 35,
      "successCount": 30,
      "failureCount": 5,
      "avgLatency": 1800,
      "totalCost": 0.08,
      "lastUsed": "2026-01-28T13:00:00Z",
      "taskTypePerformance": {
        "code-review": { "success": 18, "failure": 2 },
        "implementation": { "success": 12, "failure": 3 }
      }
    }
  },
  "routing": {
    "tierDistribution": {
      "tier1": 45,
      "tier2": 30,
      "tier3": 15,
      "tier4": 8
    },
    "decisions": [
      {
        "timestamp": "2026-01-28T15:30:00Z",
        "taskType": "implement-component",
        "complexity": "medium",
        "selectedTier": 2,
        "agent": "ui-developer",
        "result": "success",
        "cost": 0.003
      },
      {
        "timestamp": "2026-01-28T14:20:00Z",
        "taskType": "fix-styling",
        "complexity": "low",
        "selectedTier": 1,
        "agent": "ui-developer",
        "result": "success",
        "cost": 0.001
      }
    ]
  }
}
```

### Schema Field Definitions

**Agent Metrics:**
- `totalRuns`: Total task executions
- `successCount`: Tasks completed successfully
- `failureCount`: Tasks that failed or required retry
- `avgConfidence`: Rolling average of agent confidence scores (0.0-1.0)
- `lastUsed`: ISO-8601 timestamp of last execution
- `taskTypes`: Distribution of task types (understand agent specialization)
- `history`: Array of recent executions (max 100 entries, FIFO)

**Skill Metrics:**
- `activations`: Total times skill was triggered
- `lastActivated`: ISO-8601 timestamp
- `successCorrelation`: Success rate when this skill is active (0.0-1.0)
- `coActivations`: Skills frequently activated together (detect patterns)

**Model Metrics:**
- `totalRuns`: Total executions
- `successCount`/`failureCount`: Outcome tracking
- `avgLatency`: Average response time in milliseconds
- `totalCost`: Cumulative spend in USD
- `lastUsed`: ISO-8601 timestamp
- `taskTypePerformance`: Success/failure breakdown by task type

**Routing Metrics:**
- `tierDistribution`: Count of tasks routed to each tier
- `decisions`: Array of routing decisions with outcomes (max 100, FIFO)

---

## Tracking Patterns

### Pattern 1: Capturing Agent Performance

**After Agent Completes Task:**

```
Execution Flow:

1. Agent executes task
   Task: ui-developer
   Input: "Implement login form component"
   Result: Success
   Confidence: 0.90
   Duration: 45 seconds
   Tier: 2
   Model: claude