aws-containers

What this skill does

# AWS Containers

## Service Overview

| Developer Need | Recommend | Key CLI / CDK |
|---|---|---|
| Simplest container deploy (HTTP app/API, new customers) | ECS Express Mode | `aws ecs create-express-gateway-service` |
| Web app, worker, batch, scheduled task | ECS on Fargate | `aws ecs create-service` / CDK `ecsPatterns.ApplicationLoadBalancedFargateService` |
| GPU workloads or >16 vCPU | ECS on EC2 | CDK `ecs.Ec2Service` |
| Store container images | ECR | `aws ecr create-repository` |
| Web app behind a load balancer | ECS Fargate + ALB | CDK `ecsPatterns.ApplicationLoadBalancedFargateService` |
| SQS worker scaling on queue depth | ECS Fargate + SQS | CDK `ecsPatterns.QueueProcessingFargateService` |
| Cron job / scheduled task | ECS Fargate + EventBridge | CDK `ecsPatterns.ScheduledFargateTask` |
| Service mesh / service-to-service | ECS Service Connect | Configure on ECS service with Cloud Map namespace |
| Debug a running container | ECS Exec | `aws ecs execute-command --interactive --command "/bin/sh"` |

When a developer says "deploy my container" without naming a service: recommend ECS Express Mode for simple HTTP apps (replaces App Runner for new customers). Recommend ECS Fargate for everything else. Never recommend EKS unless they explicitly ask for Kubernetes.

## Overview

Provides expertise for building, deploying, and operating containerized workloads using Amazon ECS, AWS Fargate, Amazon ECR, and AWS App Runner.

**Recommended setup:** Install the AWS MCP server for sandboxed execution, audit logging, and enterprise controls. See: aws.amazon.com/mcp

**Without AWS MCP:** This skill works with any agent that has AWS CLI access. All commands use standard AWS CLI syntax.

**When NOT to use this skill:**

- Kubernetes or EKS workloads → use the kubernetes skill
- CI/CD pipeline setup for container deployments → use the deploy skill
- VPC subnet design and security group architecture → use the networking skill
- Running code without containers (Lambda, Step Functions) → use the serverless skill

**Before executing any commands:**

- You MUST verify AWS CLI v2 is installed and configured before running commands
- You MUST inform the user if required tools (AWS CLI, Docker, Session Manager plugin) are missing
- You MUST respect the user's decision to abort at any point

## Gotchas

Apply these every time. Each corrects a mistake agents make without explicit instruction.

1. **Fargate CPU/memory must be valid combinations.** Arbitrary values cause `Invalid 'cpu' setting for task`:
   - 256 (0.25 vCPU): 512 MiB, 1 GB, 2 GB
   - 512 (0.5 vCPU): 1–4 GB (1 GB increments)
   - 1024 (1 vCPU): 2–8 GB (1 GB increments)
   - 2048 (2 vCPU): 4–16 GB (1 GB increments)
   - 4096 (4 vCPU): 8–30 GB (1 GB increments)
   - 8192 (8 vCPU): 16–60 GB (4 GB increments)
   - 16384 (16 vCPU): 32–120 GB (8 GB increments)

   If the user requests an invalid combination, tell them and recommend the nearest valid option. You MUST NOT silently produce an invalid task definition.

2. **Fargate requires `awsvpc` networking mode — no exceptions.** Agents frequently suggest `bridge` or `host` mode for Fargate tasks, which causes immediate registration failure. You MUST set `networkMode` to `awsvpc` for all Fargate task definitions. On EC2, `awsvpc` is recommended; `bridge` is legacy only.

3. **Execution role vs task role — never confuse them.** `executionRoleArn`: ECS agent uses it to pull images, fetch secrets, write logs. `taskRoleArn`: application code uses it to call AWS APIs. ECS Exec permissions (`ssmmessages:*`) go on the task role. ECR pull permissions go on the execution role. `ecr:GetAuthorizationToken` MUST use `Resource: "*"` (registry-level action).

4. **Secrets are injected at task launch only — no hot-reload.** Changed secrets require `aws ecs update-service --force-new-deployment`. To reference a specific JSON key in Secrets Manager: `arn:aws:secretsmanager:region:account:secret:name-hash:json-key::` — the trailing colons are required (they represent empty version-stage and version-id fields). You can also use SSM Parameter Store with `valueFrom` pointing to the parameter ARN — the execution role needs `ssm:GetParameters` permission.

5. **ALB deregistration delay defaults to 300s — reduce to 30–60s.** This is the #1 cause of slow deployments. Set it on the target group. It SHOULD exceed your longest request duration.

6. **Set `healthCheckGracePeriodSeconds` on every ECS service behind an ALB.** Without it, the ALB marks tasks unhealthy before they're ready, the circuit breaker counts failures, and the deployment rolls back. JVM/Spring Boot apps need 60–120s.

7. **Always enable deployment circuit breaker with rollback.** Without it, bad deployments stay "in progress" for 30+ minutes. In CDK: `circuitBreaker: { rollback: true }` (specifying the property implicitly enables it; `enable` defaults to `true`).

8. **Private subnet Fargate tasks need NAT or all four VPC endpoints.** Required endpoints: `ecr.dkr` (interface), `ecr.api` (interface), `s3` (gateway — ECR stores layers in S3), `logs` (interface — for CloudWatch). The S3 gateway endpoint is the most commonly missed. For ECS Exec, also add `ssmmessages`.

9. **ECR lifecycle policies evaluate within 24 hours — not immediately.** Multi-architecture images referenced by a manifest list cannot be expired until the manifest list is deleted first. Preview before applying: first `aws ecr start-lifecycle-policy-preview --repository-name $REPO`, then `aws ecr get-lifecycle-policy-preview --repository-name $REPO --output json` to see which images would be affected.

10. **ECS Exec requires task role permissions, NOT execution role.** The task role needs `ssmmessages:CreateControlChannel`, `CreateDataChannel`, `OpenControlChannel`, `OpenDataChannel`. Tasks launched before enabling `enableExecuteCommand` do NOT support ECS Exec — force a new deployment. The container image must include the binary specified in `--command` (e.g., `/bin/sh` for interactive sessions). For command logging to S3 or CloudWatch Logs, `script` and `cat` must also be installed. Fargate platform version MUST be 1.4.0+.

11. **`awslogs` log driver mode — check your account's default.** Per [ECS docs](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/task_definition_parameters.html), the ECS service defaults to `non-blocking` mode, which drops logs when the buffer fills. The `defaultLogDriverMode` account setting can override this per account. For guaranteed log delivery (audit/compliance), explicitly set `"mode": "blocking"` in `logConfiguration.options`. Check your effective default: `aws ecs list-account-settings --name defaultLogDriverMode --effective-settings --output json`.

12. **App Runner VPC connector routes ALL application-initiated outbound traffic through the VPC.** (App Runner is sunset — new customers should use ECS Express Mode instead.) Without a NAT gateway, external API calls and AWS service calls from your application code break. App Runner's own managed traffic (pulling images, pushing logs, retrieving secrets) is NOT routed through the VPC and is unaffected. Implement retry logic with backoff for database connections at startup.

13. **For `desiredCount=1` zero-downtime deploys: `minimumHealthyPercent=100, maximumPercent=200`.** This requires capacity for 2 tasks during deployment. You MUST NOT set `minimumHealthyPercent=0` if zero downtime is required.

14. **502 Bad Gateway from ALB — check in this order:** (a) Container not listening on the port in the target group. (b) Container crashing before responding. (c) Task security group doesn't allow inbound from ALB security group on the container port. (d) Health check path returns non-200. (e) Health check timeout exceeds response time.

15. **Fargate platform version: always use `LATEST` or `1.4.0`.** Version 1.3.0 is being retired June 15, 2026 and terminated June 30, 2026.

16. **SQS worker scaling: use a custom backlog-per-task metric.** Raw `Approxim