swarm

What this skill does

# Swarm

Process many independent items in parallel. `create` builds a table handle;
`run` fans work out across rows and merges results back. One row = one unit
of work — swarm handles batching automatically.

## Flow

1. **Create.** Build a table from a source — files, a glob pattern, or
   pre-parsed records. One row per item. Returns a handle.
2. **Run.** Dispatch an `instruction` template across rows. Results are merged
   back into the table. Returns `{ completed, failed, skipped, failures }`.
3. **Aggregate.** Use `rows()` and plain JS to count, filter, or summarize.
   Do not spawn additional subagents for aggregation.
4. **Retry.** Re-run with `filter: { column: "<col>", exists: false }` to
   reprocess only failed rows.

## Choosing a source

**`glob` / `filePaths`** — one file = one row. Use when each file is an
independent unit of work. Each row gets `{ id, file }`; the subagent reads
the file itself via the `{file}` placeholder.

**`tasks`** — pass pre-built records directly. Use when the data lives inside
a file (JSONL, CSV, JSON array). Read and parse the file first inside
`eval`, then pass the records. One record = one row — do not group
multiple items into a single row.

For small files (under ~500 lines), parse and create in one block:

```javascript
const { create } = await import("@/skills/swarm");
const raw = await tools.readFile({ file_path: "/data.jsonl" });
const records = raw.trim().split("\n").map(l => JSON.parse(l));
const table = await create({ tasks: records });
console.log(table);
```

For large files, read in chunks of 500 lines to avoid truncation:

```javascript
const { create } = await import("@/skills/swarm");
let records = [];
let offset = 0;
while (true) {
  const chunk = await tools.readFile({ file_path: "/data.txt", offset, limit: 500 });
  const lines = chunk.split("\n").filter(l => l.trim());
  for (const l of lines) { records.push({ id: `r${records.length}`, text: l }); }
  if (lines.length < 500) break;
  offset += 500;
}
const table = await create({ tasks: records });
console.log(table);
```

When the file is too large to parse and dispatch in one `eval` call, split
across two blocks. Only the block that calls swarm functions needs the import:

```javascript
// eval 1: parse only — no swarm import needed
const raw = await tools.readFile({ file_path: "/data.jsonl" });
globalThis.records = raw.trim().split("\n").map(l => JSON.parse(l));
console.log(`Parsed ${globalThis.records.length} records`);
```

```javascript
// eval 2: create and dispatch
const { create, run } = await import("@/skills/swarm");
const table = await create({ tasks: globalThis.records });
const result = await run(table.id, {
  instruction: "Classify {text}",
  responseSchema: {
    type: "object",
    properties: { label: { type: "string" } },
    required: ["label"],
  },
});
console.log(result);
```

Passing `filePaths: ["/data.jsonl"]` would produce a table with **one row**
pointing at the file — not one row per record inside it.

## When to use `subagentType`

Omit `subagentType` for classification, extraction, labeling, and any task
where a single model call with structured output is sufficient. This is the
default and is significantly cheaper and faster — each dispatch is a direct
model call, no tools, no iteration.

Set `subagentType` when the task requires tools, file access, or multi-step
reasoning. Each dispatch runs a full agentic loop with the named subagent.

```javascript
// Direct model call — classification, no tools needed
await run(table.id, {
  instruction: "Classify {text}",
  responseSchema: { type: "object", properties: { label: { type: "string" } }, required: ["label"] },
});

// Subagent — needs to read files and reason over multiple steps
await run(table.id, {
  subagentType: "reviewer",
  instruction: "Review {file} for security issues.",
  responseSchema: { type: "object", properties: { finding: { type: "string" } }, required: ["finding"] },
});
```

## Instruction + context

`instruction` is a per-item template with `{column}` placeholders.
Placeholders are resolved by the framework — your column names appear in
prompts as references to the values listed alongside, never as raw
template syntax. Subagents do the work — do not process items yourself in
JS and write the results into rows.

`context` is free-form prose prepended to every subagent prompt. Use it for
shared background: domain terms, classification rules, examples, etc.

```javascript
const { create, run } = await import("@/skills/swarm");

const table = await create({ glob: "src/**/*.ts" });
const r = await run(table.id, {
  subagentType: "reviewer",
  instruction: "Review {file} for security issues. List findings or write 'no issues'.",
  context: "TypeScript Express backend using Prisma ORM. Focus on injection, auth bypass, path traversal.",
  responseSchema: {
    type: "object",
    properties: { review: { type: "string" } },
    required: ["review"],
  },
});
console.log(r);
// → { completed: 45, failed: 2, skipped: 0, failures: [...] }
```

## Structured output

`responseSchema` is required. Schema properties become top-level columns on
each row and constrain what subagents can return.

```javascript
const { run } = await import("@/skills/swarm");
await run(table.id, {
  instruction: "Classify: {text}",
  responseSchema: {
    type: "object",
    properties: {
      sentiment: { type: "string", enum: ["positive", "negative", "neutral"] },
    },
    required: ["sentiment"],
  },
});
// Row after: { id: "r1", text: "...", sentiment: "positive" }
```

## Batching

By default, swarm auto-batches to keep total dispatches under 10. For small
tables (≤10 rows) each row gets its own subagent call. For larger tables,
rows are grouped automatically.

Set `batchSize` to control grouping:

- **Number** — uniform batch size for all rows. `batchSize: 1` forces per-row
  dispatch; `batchSize: 20` groups in twenties.
- **Function** — `(row, rowCount) => number`. Returns the desired batch size
  for each row. Rows with the same batch size are grouped together, then
  chunked. Allows mixed dispatch where some rows go solo and others batch.

```javascript
const { create, run } = await import("@/skills/swarm");
const table = await create({ tasks: items });

// Complex items get individual attention; simple ones batch together
await run(table.id, {
  instruction: "Analyze {text}",
  responseSchema: {
    type: "object",
    properties: { analysis: { type: "string" } },
    required: ["analysis"],
  },
  batchSize: (row) => (row.token_count > 1000 ? 1 : 10),
});
```

Batch sizes are clamped to [1, 50] after evaluation.

## Aggregation

After `run()`, use `rows()` and plain JS — no additional subagents needed.

```javascript
const { rows } = await import("@/skills/swarm");
const data = await rows(table.id, { columns: ["sentiment"] });
const counts = {};
data.forEach(r => { counts[r.sentiment] = (counts[r.sentiment] || 0) + 1 });
console.log(counts);
// → { positive: 120, negative: 45, neutral: 35 }
```

## Chaining passes

`run` updates the table in place — chain calls to accumulate columns.

```javascript
const { create, run } = await import("@/skills/swarm");
const table = await create({ tasks: interviews });
await run(table.id, {
  instruction: "Classify sentiment of {text}",
  responseSchema: {
    type: "object",
    properties: { sentiment: { type: "string", enum: ["positive", "negative", "neutral"] } },
    required: ["sentiment"],
  },
});
await run(table.id, {
  filter: { column: "sentiment", equals: "negative" },
  instruction: "Summarize why {text} had negative sentiment.",
  responseSchema: {
    type: "object",
    properties: { summary: { type: "string" } },
    required: ["summary"],
  },
});
```

## Action-only tasks

When subagents perform actions (write a file, apply a fix) rather than return
data, use a simple schema with a status or marker field. The `exists: false`
filter still works for retries.

```javascript
const { create, run } = await impo