rlm-batch

What this skill does

# RLM Batch Processing

**You are the RLM Batch Orchestrator** - executing parallel fan-out processing where multiple sub-agents work on separate chunks of context simultaneously.

## Core Philosophy

"Divide and conquer at scale" - when a task requires processing many similar items (files, modules, documents), spawn parallel sub-agents rather than sequentially processing in a single context window.

## Your Role

You manage parallel batch execution:

1. **Parse** glob pattern and sub-prompt
2. **Match** files against pattern
3. **Estimate** cost and prompt for confirmation
4. **Spawn** sub-agents in parallel (respecting max-parallel limit)
5. **Collect** results from all sub-agents
6. **Aggregate** results according to strategy
7. **Report** final aggregated output

## Natural Language Triggers

Users may say:
- "batch process all files in src/ with: [sub-prompt]"
- "run [sub-prompt] on every file in [pattern]"
- "parallel process [pattern] to [sub-prompt]"
- "fan out [sub-prompt] across [pattern]"
- "rlm batch [pattern] [prompt]"

## Parameters

### Glob Pattern (required)
The file selection pattern. Uses standard glob syntax.

**Examples**:
- `src/**/*.ts` - All TypeScript files in src/
- `test/unit/**/*.test.js` - All unit tests
- `.aiwg/requirements/**/*.md` - All requirement docs
- `**/*.{js,ts}` - All JS and TS files recursively

### Sub-Prompt (required)
The prompt applied to each matched file independently.

**Best practices**:
- Keep prompts focused and single-purpose
- Reference the file with `{file}` placeholder
- Specify exact output format
- Make output deterministic (no random creativity)

**Good examples**:
- `"Extract all exported function names from {file}"`
- `"Count TODO comments in {file} and return as JSON: {count: N}"`
- `"Check if {file} has JSDoc comments for all exports. Return: yes/no"`

**Poor examples** (avoid these):
- `"Analyze {file}"` (too vague)
- `"Improve {file}"` (subjective, non-deterministic)
- `"Write a comprehensive report about {file}"` (unbounded output)

### --model (default: sonnet)
Which model to use for sub-agents.

**Options**:
- `opus` - Most capable, highest cost (use for complex analysis)
- `sonnet` - Balanced performance and cost (default)
- `haiku` - Fast and cheap (use for simple extraction tasks)

**Cost considerations**:
```
haiku:  ~$0.25 per 1M input tokens
sonnet: ~$3.00 per 1M input tokens
opus:   ~$15.00 per 1M input tokens
```

For 100 files @ 1k tokens each:
- haiku: ~$0.025
- sonnet: ~$0.30
- opus: ~$1.50

### --output-dir (default: .aiwg/rlm/batch-{timestamp}/)
Where to save individual sub-agent results.

Each sub-agent creates a file named after its input file:
```
.aiwg/rlm/batch-2026-02-09-1030/
├── src-auth-login.ts.result.md
├── src-auth-logout.ts.result.md
├── src-auth-refresh.ts.result.md
└── aggregate.md
```

### --aggregate (default: concat)
How to combine sub-agent results.

**Quick disambiguation** — pick by output shape:

| Sub-agent output shape | Use strategy |
|---|---|
| Independent prose findings, one per file | `concat` |
| Lists or key-value pairs likely to overlap across sub-agents | `merge` |
| Verbose findings that need executive synthesis | `summarize` |
| Findings that should be filtered to a subset (e.g., "files missing X") | `filter` (when implemented; today use `concat` + post-filter) |

**Choose deliberately** — `concat` is the default but is appropriate ONLY when sub-agent outputs are truly independent. Per Rule 6 of `rlm-context-management`, silent concatenation is the "bag of agents" anti-pattern. If sub-agents could disagree, contradict, or duplicate, use `merge` or `summarize` so the conflicts get reconciled.

**Strategies**:

#### concat (default)
Concatenate all results in order.

**Use when**: Results are independent and order matters (e.g., list of findings, one finding per file with no cross-cutting concerns).

**Output format**:
```markdown
# Batch Results

## File: src/auth/login.ts
{result from sub-agent 1}

## File: src/auth/logout.ts
{result from sub-agent 2}

...
```

#### merge
Deduplicate and merge structured results.

**Use when**: Results contain lists or key-value data with potential duplicates.

**Output format**:
```markdown
# Merged Results

Unique items across all sub-agents:
- {item1}
- {item2}
- {item3}

(Duplicates removed, sorted alphabetically)
```

**Requirements**:
- Sub-prompt MUST produce structured output (JSON, YAML, or Markdown lists)
- Deduplication based on exact string match

#### summarize
Use a final summarization agent to condense all results.

**Use when**: Individual results are verbose and need high-level synthesis.

**Process**:
1. Collect all sub-agent results
2. Spawn summarization agent with prompt:
   ```
   Summarize the following batch processing results into a concise report:

   {all results}

   Focus on:
   - Key patterns across files
   - Common issues or findings
   - Quantitative summary (counts, percentages)
   - Actionable recommendations
   ```
3. Return summarized report

**Cost note**: Adds one additional LLM call with full context of all results.

### --max-parallel (default: resolved from aiwg.config, fallback 4)
Maximum number of sub-agents running concurrently.

**Default resolution** (precedence — smallest wins, #1360):

1. **`.aiwg/aiwg.config` `parallelism.max_parallel_subagents`** — the project's provider-scoped cap (#1359). When not explicitly passed, the orchestrator uses this value as the default. Read via `aiwg config get --project parallelism.max_parallel_subagents`.
2. **RLM hard cap of 7** — values above 7 are auto-batched into sequential waves of ≤7 regardless of config.
3. **Explicit flag value** — when the user passes `--max-parallel N`, that value is the upper bound, but the cap above still wins. The orchestrator should **warn and clamp** when `N > resolved_cap`, not fail.
4. **Fallback default of 4** — when no config exists, mid-sweet-spot per REF-088.

The hardcoded 4 in earlier versions is now a **fallback**, not a primary default. Projects with `parallelism.max_parallel_subagents=10` (e.g., Codex / Copilot) will see the orchestrator use 10 by default; projects on Claude small plans (default cap=4) keep the conservative behavior automatically.

**Guidelines** (aligned with Rule 8 of `rlm-context-management`):

- Recommended range: 3-5 for most tasks, 5-7 for complex tasks
- **Hard cap: 7** — values >7 are auto-batched into sequential waves of ≤7. Per REF-086 (GRADE: LOW), independent multi-agent error amplification grows nonlinearly past the small-team coordination range
- **Context-budget interaction**: when `AIWG_CONTEXT_WINDOW` is set, the smallest of the provider cap, the budget cap, and the 7-agent hard cap applies. See `@$AIWG_ROOT/agentic/code/addons/aiwg-utils/rules/context-budget.md` Rule 6 and `@$AIWG_ROOT/agentic/code/addons/aiwg-utils/rules/subagent-scoping.md` Rule 8 for the full composition formula.

**Rate limits**:
- Claude API: 50 requests/minute
- OpenAI API: 60 requests/minute

**System resource limits**:
- Each sub-agent uses ~100MB RAM
- 7 parallel = ~700MB RAM usage
- Adjust based on available system memory

**Migration note**: Earlier versions defaulted to 10 with guidance to scale to 20-50. The lower default and hard cap are research-grounded (REF-086 LOW, REF-088 VERY LOW); for batches that genuinely need >7 parallel sub-agents, the runtime auto-batches into waves rather than rejecting the request.

## Model Selection

**Research basis**: REF-089 Appendix B (GRADE: LOW, peer-review pending) — "Qwen3-8B (non-coder) struggled without sufficient coding capabilities" and "Qwen3-235B-A22B showed smaller gains due to running out of output tokens."

| Role | Recommended | Avoid | Reason |
|---|---|---|---|
| Orchestrator (this skill) | `opus` | haiku | Emits dispatch code, parses sub-agent results, performs aggregation reasoning |
| Sub-agent — simple extraction | `haiku` | — | "Count TODOs", "list exports", yes/no checks; cheap and fast |
| Sub-agent —