joelclaw
Operate the joelclaw event bus, gateway, observability, and agent loop infrastructure via the joelclaw CLI. Use for: sending events, checking runs, starting/monitoring/cancelling agent loops, debugging failed runs, checking health, restarting the worker, inspecting step traces, managing subscriptions, gateway operations, OTEL queries, semantic recall. Triggers: 'joelclaw', 'send an event', 'check inngest', 'start a loop', 'loop status', 'why did this fail', 'debug run', 'check worker', 'restart worker', 'runs', 'what failed', 'subscribe', 'gateway', 'otel', 'recall', or any Inngest/event-bus/agent-loop task.
What this skill does
# joelclaw — CLI & Event Bus
The `joelclaw` CLI is the primary operator interface to the entire joelclaw system: event bus (Inngest), gateway, observability (OTEL), agent loops, subscriptions, and more. Built with `@effect/cli`, returns HATEOAS JSON envelopes.
**If the CLI crashes, that's the highest priority fix.**
Binary: `~/.bun/bin/joelclaw`
Source: `~/Code/joelhooks/joelclaw/packages/cli/`
Build: `bun build packages/cli/src/cli.ts --compile --outfile ~/.bun/bin/joelclaw`
## Architecture
```
┌─ Colima VM (VZ framework, aarch64) ────────────────────────────────┐
│ Talos v1.12.4 → k8s v1.35.0 (single node, namespace: joelclaw) │
│ │
│ inngest-0 StatefulSet ports 8288 (API), 8289 (dash) │
│ redis-0 StatefulSet port 6379 │
│ typesense-0 StatefulSet port 8108 │
│ restate-0 StatefulSet ports 8080, 9070, 9071 │
│ system-bus-worker Deployment port 3111 (110+ functions) │
│ restate-worker Deployment port 9080 │
│ dkron-0 StatefulSet port 8080 │
│ minio-0 StatefulSet ports 9000, 9001 │
│ docs-api Deployment port 3838 │
│ livekit-server Deployment ports 7880, 7881 │
│ bluesky-pds Deployment port 3000 │
│ │
│ ⚠️ Inngest service named inngest-svc (not inngest) │
│ k8s auto-injects INNGEST_PORT env collision otherwise │
└─────────────────────────────────────────────────────────────────────┘
↕ NodePort on localhost
Gateway daemon (always-on pi session, Redis event bridge)
NAS "three-body" (ASUSTOR, 10GbE NFS, 64TB RAID5 + NVMe cache)
Vault ~/Vault (Obsidian, PARA method — ADRs, system log, contacts)
```
**Inngest event key**: `37aa349b89692d657d276a40e0e47a15`
**k8s manifests**: `~/Code/joelhooks/joelclaw/k8s/`
## CLI Command Reference
### Health & Status
```bash
joelclaw status # Health: server + worker + k8s pods
joelclaw jobs status # Unified runtime workload snapshot (queue + Restate + Dkron + Inngest)
joelclaw inngest status # Inngest server details
joelclaw functions # List all 110+ registered functions
joelclaw refresh # Force re-register with Inngest server
```
### Send Events
```bash
joelclaw send "event/name" --data '{"key":"value"}'
joelclaw send "pipeline/video.requested" --data '{"url":"https://youtube.com/watch?v=XXX"}'
joelclaw send "agent/story.start" --data '{"prdPath":"/abs/path/prd.json","storyId":"S-1"}'
```
### View Runs
```bash
joelclaw runs # Recent 10
joelclaw runs --count 20 --hours 24 # More runs, wider window
joelclaw runs --status FAILED # Just failures
joelclaw run <RUN_ID> # Step trace + errors for one run
```
### View Events
```bash
joelclaw events # Last 4 hours
joelclaw events --prefix memory/ --hours 24
joelclaw events --prefix agent/ --hours 24
joelclaw events --count 50 --hours 48
```
### Logs
```bash
joelclaw logs # Worker stdout (default 30 lines)
joelclaw logs errors # Worker stderr (stack traces)
joelclaw logs server # Inngest k8s pod logs
joelclaw logs server -n 50 --grep error # Filtered server errors
joelclaw logs worker --grep "observe" # Grep worker logs
```
### Structured Log Writes
```bash
joelclaw log write --action configure --tool cli --detail "updated capability adapter config" --reason "ADR-0169 phase 1"
```
`log` writes canonical structured entries (slog backend). `logs` remains runtime log read/analyze.
### Secrets
```bash
joelclaw secrets status
joelclaw secrets lease <name> --ttl 15m
joelclaw secrets revoke <lease-id>
joelclaw secrets revoke --all
joelclaw secrets audit --tail 50
joelclaw secrets env --dry-run
```
### Notify
```bash
joelclaw notify send "Worker restarted and healthy" --priority normal
joelclaw notify send "Immediate action required" --priority urgent --telegram-only
```
### Capability Adapter Paths (ADR-0169 phase 4)
`mail`, `otel`, `recall`, and `subscribe` now run through the CLI capability registry/adapter runtime while preserving their existing command UX and JSON envelopes.
### Deploy
```bash
joelclaw deploy worker # dry-run deploy plan
joelclaw deploy worker --restart --execute # execute worker sync deployment
joelclaw deploy worker --restart --execute --force # force with active runs (disruptive)
```
### Heal
```bash
joelclaw heal list
joelclaw heal run RUN_FAILED --phase fix --context '{"run-id":"01ABC"}' # dry-run
joelclaw heal run RUN_FAILED --phase fix --context '{"run-id":"01ABC"}' --execute # execute
```
### Gateway
```bash
joelclaw gateway status # Gateway health + session info
joelclaw gateway events # Recent gateway events
joelclaw gateway test # Send test message through gateway
joelclaw gateway restart # Restart gateway daemon
joelclaw gateway stream # Live stream gateway events
```
### Workload Rig
```bash
joelclaw workload plan "intent" --stages-from stages.json --write-plan plan.json
joelclaw workload run plan.json --stage <stage-id> --tool pi|codex|claude
joelclaw workload dispatch plan.json --to BlueFox --from MaroonReef --send-mail
joelclaw workload sandboxes list
joelclaw workload sandboxes cleanup --expired --dry-run
joelclaw workload sandboxes janitor
```
Runtime truth:
- `workload run` admits into Redis, then Restate executes the stage through `dagOrchestrator` → `dagWorker`
- `dagWorker` handlers: `shell`, `infer`, `microvm`
- `infer` now runs `pi` inside the `restate-worker` k8s pod with mounted auth, skills, and identity
- `--stages-from` is the explicit stage-DAG front door; dependency gates and cycle detection are real
### Async runtime monitoring in pi
The loaded pi extension at `packages/pi-extensions/inngest-monitor/index.ts` now exposes `runtime_jobs_monitor` alongside `inngest_send` / `inngest_runs`.
Use it when you want a pi session to keep watch on the ADR-0217 runtime substrate while you do other work:
- `runtime_jobs_monitor {"action":"start","interval":5,"report":true}` — start background polling of `joelclaw jobs status`
- `runtime_jobs_monitor {"action":"status"}` — inspect the latest runtime snapshot
- `runtime_jobs_monitor {"action":"stop"}` — stop the poller and send a final follow-up summary
The widget shows runtime health first (queue / Restate / Dkron / Inngest), then any followed Inngest runs underneath. Severity changes and meaningful workload-state changes emit OTEL and hidden follow-up messages for asynchronous report-back, so a healthy-but-now-held backlog does not stay silent.
### Observability (OTEL)
```bash
joelclaw otel list --hours 1 # Recent telemetry events
joelclaw otel search "error" --hours 24 # Search OTEL events
joelclaw otel stats --hours 24 # Aggregate stats
joelclaw otel emit "action.name" --source codex --component agent-loop --success # Emit event
```
### System Knowledge (ADR-0199)
```bash
joelclaw knowledge search "query" # Search system knowledge
joelclaw knowledge search "query" --type adr # Filter by type (adr|skill|lesson|pattern|retro|failed_target)
joelclaw knowledge sync # Re-index ADRs + skills from filesystem
joelclaw knowledge clear-failed <target> # Clear resolved failed targets
```
Brain/codebase patterns (browsable by agents):
```bash
ls ~/Vault/system/brain/codebase/ # List estaRelated in AI Agents
skill-development
IncludedComprehensive meta-skill for creating, managing, validating, auditing, and distributing Claude Code skills and slash commands (unified in v2.1.3+). Provides skill templates, creation workflows, validation patterns, audit checklists, naming conventions, YAML frontmatter guidance, progressive disclosure examples, and best practices lookup. Use when creating new skills, validating existing skills, auditing skill quality, understanding skill architecture, needing skill templates, learning about YAML frontmatter requirements, progressive disclosure patterns, tool restrictions (allowed-tools), skill composition, skill naming conventions, troubleshooting skill activation issues, creating custom slash commands, configuring command frontmatter, using command arguments ($ARGUMENTS, $1, $2), bash execution in commands, file references in commands, command namespacing, plugin commands, MCP slash commands, Skill tool configuration, or deciding between skills vs slash commands. Delegates to docs-management skill for official documentation.
reprompter
IncludedTransform messy prompts into well-structured, effective prompts — single or multi-agent. Use when: "reprompt", "reprompt this", "clean up this prompt", "structure my prompt", rough text needing XML tags and best practices, "reprompter teams", "repromptception", "run with quality", "smart run", "smart agents", multi-agent tasks, audits, parallel work, anything going to agent teams. Don't use when: simple Q&A, pure chat, immediate execution-only tasks. See "Don't Use When" section for details. Outputs: Structured XML/Markdown prompt, quality score (before/after), optional team brief + per-agent sub-prompts, agent team output files. Success criteria: Single mode quality score ≥ 7/10; Repromptception per-agent prompt quality score 8+/10; all required sections present, actionable and specific.
adaptive-compaction
IncludedAdaptive add-on policy and recovery layer that decides WHEN to compact, prune, snapshot, or fork -- replacing fixed-percent auto-compaction across Claude Code, Codex, and MCP-capable hosts. Trigger on auto-compact timing or damage: "when should I compact", "is it safe to compact now or start a fresh session", "auto-compact fires too early/mid-task", "switching to an unrelated task but the window still has space", "context rot", "answers get worse the longer the session runs", "the agent forgot the plan or my decisions after it summarized", "add a layer on top that manages context without changing the agent", raising autoCompactWindow to give the policy room, or installing/tuning a cross-tool compaction policy or PreCompact hook -- even when "compaction" is never said but the problem is context-window pressure or post-summarization memory loss. Do NOT use to summarize a conversation, build RAG, write a summarization prompt (decides WHEN not HOW), or answer max-context-length trivia.
agent-skill-creator
IncludedCreate cross-platform agent skills from workflow descriptions. Activates when users ask to create an agent, automate a repetitive workflow, create a custom skill, or need advanced agent creation. Triggers on phrases like create agent for, automate workflow, create skill for, every day I have to, daily I need to, turn process into agent, need to automate, create a cross-platform skill, validate this skill, export this skill, migrate this skill. Supports single skills, multi-agent suites, transcript processing, template-based creation, interactive configuration, cross-platform export, and spec validation.
llm-wiki
IncludedUse when building or maintaining a persistent personal knowledge base (second brain) in Obsidian where an LLM incrementally ingests sources, updates entity/concept pages, maintains cross-references, and keeps a synthesis current. Triggers include "second brain", "Obsidian wiki", "personal knowledge management", "ingest this paper/article/book", "build a research wiki", "compound knowledge", "Memex", or whenever the user wants knowledge to accumulate across sessions instead of being re-derived by RAG on every query.
skill-master
IncludedAgent Skills authoring, evaluation, and optimization. Create, edit, validate, benchmark, and improve skills following the agentskills.io specification. Use when designing SKILL.md files, structuring skill folders (references, scripts, assets), ingesting external documentation into skills, running trigger evals, benchmarking skill quality, optimizing descriptions, or performing blind A/B comparisons. Keywords: agentskills.io, SKILL.md, skill authoring, eval, benchmark, trigger optimization.