when-optimizing-agent-learning-use-reasoningbank-intelligence

What this skill does

# ReasoningBank Intelligence - Adaptive Agent Learning

## Overview

Implement adaptive learning with ReasoningBank for pattern recognition, strategy optimization, and continuous improvement. Use when building self-learning agents, optimizing decision-making, or implementing meta-cognitive systems.

## When to Use

- Agent performance needs improvement
- Repetitive tasks require optimization
- Need pattern recognition from experience
- Strategy refinement through learning
- Building self-improving systems
- Meta-cognitive capabilities needed

## Theoretical Foundation

### ReasoningBank Architecture

1. **Trajectory Tracking**: Record decision paths and outcomes
2. **Verdict Judgment**: Evaluate success/failure of strategies
3. **Memory Distillation**: Extract patterns from experience
4. **Pattern Recognition**: Identify successful approaches
5. **Strategy Optimization**: Apply learned patterns to new situations

### AgentDB Integration (Optional)

- 150x faster vector operations
- HNSW indexing for similarity search
- Quantization for memory efficiency
- Batch operations for performance

## Phase 1: Initialize Learning System (10 min)

### Objective
Set up ReasoningBank with trajectory tracking

### Agent: ML-Developer

**Step 1.1: Initialize ReasoningBank**
```javascript
const ReasoningBank = require('reasoningbank');

const learningSystem = new ReasoningBank({
  storage: {
    type: 'agentdb', // Or 'memory', 'disk'
    path: './reasoning-bank-data',
    quantization: 'int8' // 4-32x memory reduction
  },
  indexing: {
    enabled: true,
    type: 'hnsw', // 150x faster search
    dimensions: 768
  },
  learning: {
    algorithm: 'decision-transformer',
    learningRate: 0.001,
    batchSize: 32
  }
});

await learningSystem.init();
await memory.store('reasoningbank/system', learningSystem.config);
```

**Step 1.2: Define Trajectory Schema**
```javascript
const trajectorySchema = {
  id: 'uuid',
  timestamp: 'datetime',
  context: {
    task: 'string',
    environment: 'object',
    constraints: 'array'
  },
  reasoning: [
    {
      step: 'number',
      thought: 'string',
      action: 'string',
      observation: 'string'
    }
  ],
  outcome: {
    success: 'boolean',
    metrics: 'object',
    verdict: 'string'
  }
};

await learningSystem.registerSchema('trajectory', trajectorySchema);
```

**Step 1.3: Configure Verdict Criteria**
```javascript
const verdictCriteria = {
  success: {
    thresholds: {
      performance: 0.8,
      efficiency: 0.75,
      quality: 0.9
    },
    weights: {
      performance: 0.4,
      efficiency: 0.3,
      quality: 0.3
    }
  },
  failure: {
    reasons: [
      'timeout',
      'error',
      'poor_quality',
      'resource_exhaustion'
    ]
  }
};

await learningSystem.configureVerdicts(verdictCriteria);
```

### Validation Criteria
- [ ] ReasoningBank initialized
- [ ] Trajectory schema registered
- [ ] Verdict criteria configured
- [ ] Storage backend ready

### Hooks Integration
```bash
npx claude-flow@alpha hooks pre-task \
  --description "Initialize ReasoningBank learning system" \
  --complexity "high"

npx claude-flow@alpha hooks post-task \
  --task-id "reasoningbank-init" \
  --memory-key "reasoningbank/initialization"
```

## Phase 2: Capture Patterns (10 min)

### Objective
Record agent decisions and outcomes for learning

### Agent: SAFLA-Neural

**Step 2.1: Track Trajectories**
```javascript
async function trackTrajectory(task, agent) {
  const trajectory = {
    id: generateUUID(),
    timestamp: new Date(),
    context: {
      task: task.description,
      environment: getEnvironment(),
      constraints: task.constraints
    },
    reasoning: []
  };

  // Hook into agent execution
  agent.on('thought', (thought) => {
    trajectory.reasoning.push({
      step: trajectory.reasoning.length + 1,
      thought: thought.text,
      action: null,
      observation: null
    });
  });

  agent.on('action', (action) => {
    const lastStep = trajectory.reasoning[trajectory.reasoning.length - 1];
    lastStep.action = action.description;
  });

  agent.on('observation', (obs) => {
    const lastStep = trajectory.reasoning[trajectory.reasoning.length - 1];
    lastStep.observation = obs.result;
  });

  agent.on('complete', async (result) => {
    trajectory.outcome = {
      success: result.success,
      metrics: result.metrics,
      verdict: await evaluateVerdict(result)
    };

    await learningSystem.storeTrajectory(trajectory);
  });

  return trajectory;
}
```

**Step 2.2: Evaluate Verdicts**
```javascript
async function evaluateVerdict(result) {
  const scores = {
    performance: result.metrics.score,
    efficiency: result.metrics.duration / result.metrics.expectedDuration,
    quality: result.metrics.qualityScore
  };

  const weightedScore = Object.keys(scores).reduce((sum, key) => {
    return sum + scores[key] * verdictCriteria.success.weights[key];
  }, 0);

  const verdict = {
    score: weightedScore,
    passed: weightedScore >= Object.values(verdictCriteria.success.thresholds)
      .reduce((sum, t) => sum + t, 0) / 3,
    breakdown: scores,
    reasoning: generateVerdictReasoning(scores, weightedScore)
  };

  await learningSystem.recordVerdict(result.id, verdict);
  return verdict;
}
```

**Step 2.3: Pattern Extraction**
```javascript
async function extractPatterns() {
  // Get all successful trajectories
  const successfulTrajectories = await learningSystem.query({
    'outcome.verdict.passed': true
  });

  // Extract common patterns using AgentDB vector similarity
  const patterns = await learningSystem.analyzePatterns({
    trajectories: successfulTrajectories,
    similarity: {
      method: 'cosine',
      threshold: 0.85,
      index: 'hnsw' // 150x faster
    },
    clustering: {
      algorithm: 'dbscan',
      minSamples: 3,
      epsilon: 0.15
    }
  });

  await memory.store('reasoningbank/patterns', patterns);
  return patterns;
}
```

### Validation Criteria
- [ ] Trajectories captured
- [ ] Verdicts evaluated
- [ ] Patterns extracted
- [ ] Similarity clustering complete

## Phase 3: Optimize Strategies (10 min)

### Objective
Apply learned patterns to improve future decisions

### Agent: Performance-Analyzer

**Step 3.1: Train Decision Model**
```javascript
async function trainDecisionModel(patterns) {
  // Use Decision Transformer (from ReasoningBank's 9 RL algorithms)
  const model = await learningSystem.createModel({
    algorithm: 'decision-transformer',
    config: {
      hiddenSize: 256,
      numLayers: 4,
      numHeads: 8,
      maxTrajectoryLength: 50,
      learningRate: 0.0001
    }
  });

  // Prepare training data from successful patterns
  const trainingData = patterns.map(pattern => ({
    states: pattern.reasoning.map(r => r.thought),
    actions: pattern.reasoning.map(r => r.action),
    rewards: calculateRewards(pattern.outcome),
    returns: calculateReturnsToGo(pattern.outcome)
  }));

  // Train with batch operations (AgentDB optimization)
  await model.train({
    data: trainingData,
    epochs: 100,
    batchSize: 32,
    validation: 0.2,
    callbacks: {
      onEpoch: (epoch, metrics) => {
        console.log(`Epoch ${epoch}: loss=${metrics.loss}, accuracy=${metrics.accuracy}`);
      }
    }
  });

  await learningSystem.saveModel('decision-model', model);
  return model;
}
```

**Step 3.2: Generate Strategy Recommendations**
```javascript
async function generateRecommendations() {
  const patterns = await memory.retrieve('reasoningbank/patterns');

  const recommendations = patterns.map(pattern => {
    const frequency = pattern.instances.length;
    const avgScore = pattern.instances.reduce((sum, i) =>
      sum + i.outcome.verdict.score, 0) / frequency;

    return {
      pattern: {
        description: summarizePattern(pattern),
        reasoning: pattern.commonReasoning,
        actions: pattern.commonActions
      },
      metrics: {
        frequency,
        avgScore,
        consistency: calculateConsistency(pattern.