agents-debug

What this skill does

# debug

Diagnose why your AgentCore agent or environment isn't working correctly.

## When to use

- Your agent is returning wrong answers or errors
- Tool calls are failing or timing out
- Agent works locally but fails after deploying
- Logs aren't showing up in CloudWatch
- The AgentCore CLI isn't working or environment seems broken
- `agentcore` command not found or prerequisites are missing

Do NOT use for:

- Deploy failures (CDK errors, IAM during deploy) → use `agents-deploy`
- Scaffolding a new project → use `agents-get-started`
- Measuring quality or setting up monitoring → use `agents-optimize`

## Input

`$ARGUMENTS` is optional:

```
/agents-debug                      # interactive — describe what's wrong
/agents-debug traces               # read and explain recent traces
/agents-debug logs                 # search recent logs for errors
/agents-debug memory               # diagnose memory recall issues specifically
/agents-debug doctor               # check environment prerequisites
```

## Process

### Step 0: Determine problem type

If the developer's issue is about the CLI itself (command not found, prerequisites, environment setup), load [`references/doctor.md`](references/doctor.md) and follow its diagnostic checklist.

If the issue is about agent behavior (wrong answers, errors, timeouts, tool failures), continue with Step 1 below.

### Step 1: Verify CLI version

Run `agentcore --version`. This skill requires v0.9.0 or later. If the version is older, tell the developer to run `agentcore update` before proceeding.

### Step 2: Understand the symptom

Ask (or infer from context):

> "What's happening?
>
> 1. The agent returns an error message
> 2. The agent returns a wrong or unhelpful answer
> 3. A specific tool call is failing
> 4. Memory isn't working (agent doesn't remember things)
> 5. The agent is slow or timing out
> 6. I want to understand what the agent did in a specific session"

### Step 3: Read traces and logs automatically

Don't ask the developer to paste logs — read them directly.

```bash
# List recent traces
agentcore traces list --runtime <AgentName> --since 1h

# Get the most recent trace ID
agentcore traces list --runtime <AgentName> --since 1h --limit 1

# Download and read the trace
agentcore traces get <traceId> --runtime <AgentName>

# Search logs for errors
agentcore logs --runtime <AgentName> --since 1h --level error

# Search logs for a specific pattern
agentcore logs --runtime <AgentName> --since 2h --query "timeout"
agentcore logs --runtime <AgentName> --since 2h --query "model access"
```

**Important:** CloudWatch put-to-get latency is **~10 seconds end-to-end** — that's the delay from when a span is emitted to when it's readable by `agentcore traces get` or `agentcore run eval`. There is **no separate "trace ingested but eval not ready yet" window**; the same ingestion step unlocks both paths. Older skills and docs said 30–60s for traces and 2–5 minutes for evals — both are stale. If you just invoked the agent, wait ~15 seconds and both trace reads and evals will work.

Read `agentcore/agentcore.json` to get the agent name if not provided.

### Step 4: Diagnose by symptom

---

## Symptom: "model access denied" or model error

**Most common cause:** The model isn't enabled in the Bedrock console for your region.

Fix:

1. Go to AWS Console → Amazon Bedrock → Model access
2. Enable the model your agent uses
3. Wait 1–2 minutes for access to propagate

**Second cause:** The execution role is missing `bedrock:InvokeModel`.

Check:

```bash
aws iam simulate-principal-policy \
  --policy-source-arn $(agentcore status --json | jq -r '.runtimes[0].executionRoleArn') \
  --action-names bedrock:InvokeModel \
  --resource-arns "arn:aws:bedrock:*::foundation-model/*"
```

**Third cause:** Cross-region inference profile requires model access in all regions.

Model IDs starting with a geographic prefix are cross-region inference profiles that route requests within that geography:

| Prefix | Geography | Example destination regions |
|---|---|---|
| `us.` | United States | us-east-1, us-east-2, us-west-2 |
| `eu.` | Europe | eu-central-1, eu-west-1, eu-west-2, eu-west-3 |
| `apac.` | Asia Pacific | ap-northeast-1, ap-southeast-1, ap-southeast-2, ap-south-1 |
| `global.` | All commercial regions worldwide | All supported regions |

The AgentCore CLI scaffolds `global.` by default (e.g., `global.anthropic.claude-sonnet-4-5-20250929-v1:0`). All prefixes require model access enabled in every destination region the profile covers. For `us.` profiles, enable in all US regions; for `eu.`, all EU regions; for `global.`, all supported regions. Not all models support all prefixes — `global.` is currently available for select models only. Use `global.` for maximum throughput when available, or a geographic prefix when data residency requirements constrain where inference can run. Check the Bedrock inference profiles docs for current model × prefix availability.

---

## Symptom: Tool call failing

**Step 1:** Find the failing tool call in the trace:

```bash
agentcore traces get <traceId> --runtime <AgentName>
```

Look for tool call entries with error status.

**Step 2:** Check the gateway status:

```bash
agentcore status --type gateway
agentcore fetch access --name <AgentName> --type agent
```

**Step 3:** Common tool call failures:

**Gateway URL not set (local dev):**
The `AGENTCORE_GATEWAY_*_URL` env var is only set after deploy. In `agentcore dev`, gateway tools aren't available. This is expected — the agent should handle this gracefully.

**Auth failure on tool call:**

```bash
agentcore logs --runtime <AgentName> --since 1h --query "auth"
```

Check that the credential is configured correctly: `agentcore status --type credential`

**Lambda function error:**
The Lambda itself is failing. Check Lambda logs directly:

```bash
aws logs tail /aws/lambda/<function-name> --since 1h
```

**Policy denial:**
If a policy engine is attached, check policy decision logs:

```bash
agentcore logs --runtime <AgentName> --since 1h --query "policy"
agentcore status --type policy-engine
```

---

## Symptom: Wrong or unhelpful answers

**Step 1:** Read the trace to see the agent's reasoning:

```bash
agentcore traces get <traceId> --runtime <AgentName>
```

The trace shows the model's reasoning steps, tool calls made, and the final response. Look for:

- Did the agent use the right tools?
- Did the tool calls return the expected data?
- Is the system prompt providing the right context?

**Step 2:** Check if memory is involved:
If the agent should be using memory context but isn't, see the "Symptom: Memory not persisting" section later in this skill, or load [`references/doctor.md`](references/doctor.md) if this is an environment issue.

**Step 3:** Common causes:

- System prompt is too vague or missing key context
- Agent isn't calling the right tools (tool descriptions need improvement)
- Tool is returning unexpected data format
- Model ID is wrong for the task (e.g., using a smaller model for complex reasoning)

---

## Symptom: Memory not working

**Memory not persisting across sessions (LTM):**

1. Verify LTM strategies are configured (SEMANTIC or USER_PREFERENCE):

```bash
agentcore status --type memory --json | jq '.memories[].strategies'
```

1. Wait 5–30 seconds after a session ends — LTM extraction is async. The agent must finish its session before facts are extracted.

2. Use UUIDs (v4) for session IDs — the platform requires a minimum of 33 characters. Short IDs like "session-1" cause LTM to fail silently. `agentcore invoke` generates compliant IDs by default.

3. Verify the memory resource is ACTIVE:

```bash
agentcore status --type memory
```

**Memory not loading at session start:**

1. Check the `MEMORY_*_ID` env var is set:

```bash
agentcore status --type memory --json | jq '.memories[].id'
```

1. Verify the `actor_id` is consistent across sessions — memory is scoped per actor.

2. Check the namespace paths in you