awesome-harness-engineering

What this skill does

```markdown
---
name: awesome-harness-engineering
description: Curated knowledge base for harness engineering — the practice of shaping the environment around AI agents for reliability, context management, evaluation, and safe autonomy.
triggers:
  - harness engineering for AI agents
  - how to build an agent harness
  - context engineering for coding agents
  - agent evaluation and observability
  - safe autonomy and guardrails for agents
  - AGENTS.md or CLAUDE.md setup
  - benchmarking agent harness quality
  - long-running agent workflow design
---

# Awesome Harness Engineering

> Skill by [ara.so](https://ara.so) — Daily 2026 Skills collection.

## What Is Harness Engineering?

Harness engineering is the discipline of shaping the **environment around an AI agent** so it can work reliably. The harness is everything except the model itself:

- **Context & memory management** — what the agent sees and remembers
- **Constraints & guardrails** — what the agent is allowed to do
- **Specs & agent files** — repo-local instructions that persist across sessions
- **Evals & observability** — how you measure and debug agent behavior
- **Orchestration & runtime** — how tasks are scheduled, retried, and handed off

The key insight: *weak results from coding agents are usually harness problems, not model problems.*

---

## Core Mental Model

```
┌─────────────────────────────────────────┐
│              AGENT HARNESS              │
│                                         │
│  ┌─────────┐   ┌──────────────────┐    │
│  │ Prompts │   │  Context Window  │    │
│  │ & Specs │   │  (working memory)│    │
│  └─────────┘   └──────────────────┘    │
│                                         │
│  ┌──────────┐  ┌──────────────────┐    │
│  │  Tools   │  │  Guardrails &    │    │
│  │ (bounded)│  │  Sandboxing      │    │
│  └──────────┘  └──────────────────┘    │
│                                         │
│  ┌──────────────────────────────────┐  │
│  │  Evals, Traces & Observability   │  │
│  └──────────────────────────────────┘  │
│                                         │
│              ┌───────┐                 │
│              │ MODEL │                 │
│              └───────┘                 │
└─────────────────────────────────────────┘
```

---

## 1. Repo-Local Agent Instructions

### AGENTS.md / CLAUDE.md / agent.md

Place a file in the root of your repo to give agents persistent, repo-scoped instructions.

**`AGENTS.md` (open format — works with many agents):**

```markdown
# Agent Instructions

## Repo Overview
This is a TypeScript monorepo. Packages are under `packages/`.

## Development Commands
- `pnpm install` — install dependencies
- `pnpm build` — build all packages
- `pnpm test` — run all tests
- `pnpm lint` — lint all packages

## Coding Conventions
- Use named exports only
- All async functions must handle errors explicitly
- Write tests in Vitest; place them next to the source file as `*.test.ts`

## What NOT to Do
- Do not modify `packages/core/src/generated/` — these are auto-generated
- Do not commit `.env` files
- Do not use `any` in TypeScript

## Verification Steps
After making changes, always run:
1. `pnpm build`
2. `pnpm test`
3. `pnpm lint`
```

**`CLAUDE.md` (Claude Code specific):**

```markdown
# Claude Instructions

## Project Context
REST API service using Express + Prisma + PostgreSQL.

## Environment Setup
\`\`\`bash
cp .env.example .env
# Set DATABASE_URL in .env
pnpm install
pnpm db:migrate
\`\`\`

## Key Patterns
- Route handlers go in `src/routes/`
- Business logic goes in `src/services/`
- Database queries go in `src/repositories/`
- Always validate input with Zod schemas in `src/schemas/`

## Testing
- Unit tests: `pnpm test:unit`
- Integration tests (requires DB): `pnpm test:integration`
- Check coverage: `pnpm test:coverage`

## Agent Checkpoints
After each feature, verify:
- [ ] Types compile: `pnpm typecheck`
- [ ] Tests pass: `pnpm test`
- [ ] No lint errors: `pnpm lint`
```

---

## 2. Context Engineering Patterns

### Treat Context as a Budget

```python
# Pattern: Context budget management
# Don't dump everything — be selective about what enters context

CONTEXT_BUDGET = {
    "system_prompt": 2000,      # tokens — keep it tight
    "task_spec": 3000,          # the current task description
    "relevant_files": 20000,    # only files the agent needs RIGHT NOW
    "tool_results": 10000,      # recent tool outputs (rolling window)
    "conversation": 5000,       # recent turns only
    "reserve": 10000,           # leave room for model output
}

# Rolling window for tool results — don't let noisy output fill context
def add_tool_result(context_window: list, result: str, max_tokens: int = 2000):
    """Add tool result, truncating if needed, keeping failures visible."""
    truncated = result[:max_tokens] if len(result) > max_tokens else result
    context_window.append({
        "role": "tool",
        "content": truncated,
        "truncated": len(result) > max_tokens
    })
    return context_window
```

### KV-Cache Locality (from Manus playbook)

```python
# Pattern: Keep stable content at the TOP of context to maximize cache hits
# System prompt (never changes) → repo instructions (rarely changes) → task state → recent actions

def build_context(system_prompt, repo_instructions, task_state, recent_actions):
    """
    Order matters for KV-cache efficiency.
    Stable content first = more cache hits = lower latency + cost.
    """
    return [
        {"role": "system", "content": system_prompt},          # STABLE — cache hit
        {"role": "user", "content": repo_instructions},        # STABLE — cache hit  
        {"role": "user", "content": f"Current task:\n{task_state}"},   # changes per task
        *recent_actions,                                        # changes every step
    ]
```

### Filesystem as External Memory

```python
# Pattern: Use files to offload memory across context windows
import json
from pathlib import Path
from datetime import datetime

class AgentWorkingState:
    """Persist agent state to filesystem so it survives context resets."""
    
    def __init__(self, workspace: Path):
        self.workspace = workspace
        self.state_file = workspace / ".agent_state.json"
    
    def save(self, state: dict):
        state["updated_at"] = datetime.utcnow().isoformat()
        self.state_file.write_text(json.dumps(state, indent=2))
    
    def load(self) -> dict:
        if not self.state_file.exists():
            return {"tasks": [], "completed": [], "notes": []}
        return json.loads(self.state_file.read_text())
    
    def add_note(self, note: str):
        """Agent can leave notes for itself across context windows."""
        state = self.load()
        state["notes"].append({"note": note, "at": datetime.utcnow().isoformat()})
        self.save(state)
    
    def mark_complete(self, task_id: str, artifact_path: str):
        state = self.load()
        state["completed"].append({
            "task_id": task_id,
            "artifact": artifact_path,
            "at": datetime.utcnow().isoformat()
        })
        self.save(state)
```

---

## 3. Initializer Agent Pattern (Anthropic)

For long-running tasks, use a lightweight **initializer agent** to set up the harness before the main agent runs:

```python
# Pattern: Initializer agent creates the harness for the worker agent

INITIALIZER_PROMPT = """
You are an initializer agent. Your job is to:
1. Read the feature request
2. Create a feature list file: .agent/feature_list.md
3. Create an init script: .agent/init.sh  
4. Create a verification script: .agent/verify.sh
5. Write a handoff summary: .agent/handoff.md

Do NOT implement the feature. Only set up the harness.
"""

WORKER_PROMPT_TEMPLATE = """
You are a coding agent. The initializer has prepared your harness.

Before starting, run: bash .agent/init.sh
Your tasks are in: .agent/feature_list.md

After each task:
1. Run bash .agent/verify.sh
2. If verification pass