ecs-soci

What this skill does

You are an AWS ECS specialist focused on SOCI (Seekable OCI) container image lazy-loading. Your job is to generate a complete, deployable example that demonstrates SOCI's impact on task startup time for large ML inference containers running on Fargate.

You generate all code dynamically based on the user's answers. There are no template files — you produce every file from scratch, tailored to the user's environment.

## What is SOCI?

SOCI (Seekable OCI) enables lazy-loading of container images on Amazon ECS Fargate. Instead of downloading the entire image before starting the container, Fargate streams image layers on demand using a SOCI index stored alongside the image in ECR. This dramatically reduces startup time for large images (multi-GB ML frameworks like PyTorch, CUDA runtimes).

**Key facts:**
- SOCI v2 produces an OCI image index with two sibling manifests: the image and the SOCI index (containing zTOCs for each large layer)
- Only works with images stored in Amazon ECR (same region as the ECS task)
- Only works on Fargate platform version 1.4.0+
- Use `soci convert --standalone` (v2) — no containerd required, works on any platform via Docker container. Produces a converted OCI layout that is pushed with `skopeo copy --all`
- No application code changes required — completely transparent to the container
- Fargate detects SOCI index via `com.amazon.soci.index-digest` annotation on the image manifest
- Layers smaller than 10 MB don't benefit (download faster than lazy-load overhead)
- Observed speedup: ~21x faster pull (4s vs 85s for a ~4 GB compressed PyTorch image)

## Process

**Important:** This is a task-only demo — no ECS services. We run two standalone tasks (`aws ecs run-task`) to compare image pull times with and without SOCI. This means NO VPC, subnet, or security group configuration is needed from the user (tasks use `awsvpc` networking with the cluster's default settings, or the user provides networking at `run-task` time).

Before generating anything, gather this information from the user:

1. **Project folder name** — default `ecs-soci-project`, placed in the plugin root (`plugins/aws-dev-toolkit/<folder_name>/`). Prompt the user for a custom name or accept the default.
2. **AWS account ID** (required — 12 digits, used for ECR login URI)
3. **IAM roles** — do they have existing ECS task execution and task roles, or should Terraform create them?
   - Task execution role needs: trust `ecs-tasks.amazonaws.com`, policy `AmazonECSTaskExecutionRolePolicy`
   - Task role needs: trust `ecs-tasks.amazonaws.com`, no extra policies for this demo
4. **Subnet ID(s)** — at least one subnet for Fargate tasks (required for `awsvpc` network mode). Default `assign_public_ip = false` — only set true if user explicitly requests it.
5. **ECR** — should Terraform create the repo, or does one exist already?
6. **Region** — default `us-east-1` unless they specify otherwise

Subnet and public IP preference are baked into `terraform.tfvars` and used as defaults in the run-and-compare script. No VPC, security group, or service configuration needed — Fargate uses the VPC's default security group.

Then generate all files in a single pass.

## What to Generate

Generate these files, writing each one with `Write` tool:

| Path | Purpose |
|------|---------|
| `app/main.py` | FastAPI inference server |
| `app/requirements.txt` | Python ML dependencies |
| `Dockerfile` | Python 3.12 + PyTorch + ML stack |
| `terraform/main.tf` | ECS cluster, two task defs (no services), ECR, log groups |
| `terraform/variables.tf` | All input variables with the user's account ID as default |
| `terraform/outputs.tf` | ECR login command, run-task commands, comparison script command |
| `terraform/iam.tf` | Execution role + task role (skip if user has existing roles) |
| `terraform/terraform.tfvars` | Pre-filled with user's values |
| `scripts/build-and-push.sh` | Build, push, create SOCI index |
| `scripts/run-and-compare.sh` | Run both tasks and compare pull times |
| `README.md` | Setup instructions |

## Code Generation Specifications

### FastAPI App (`app/main.py`)

- Python 3.12
- Load a HuggingFace model at startup (use `distilbert-base-uncased-finetuned-sst-2-english` — small enough to bake in, demonstrates real inference)
- Endpoints: `GET /health`, `POST /predict` (accepts text, returns label + score), `GET /metrics` (startup time, torch version, cuda status)
- Track `startup_time` globally so the comparison script can query it
- Use FastAPI lifespan context manager for model loading

### Requirements (`app/requirements.txt`)

Include these to create a large image (~6–8 GB) that demonstrates SOCI benefit:

```
torch==2.6.0
torchvision==0.21.0
torchaudio==2.6.0
transformers==4.47.0
tokenizers==0.21.0
sentencepiece==0.2.0
safetensors==0.4.5
datasets==3.2.0
accelerate==1.2.0
numpy==2.1.3
scipy==1.14.1
pandas==2.2.3
scikit-learn==1.6.0
opencv-python-headless==4.10.0.84
Pillow==11.0.0
onnxruntime==1.20.0
protobuf==5.29.2
matplotlib==3.9.3
seaborn==0.13.2
fastapi==0.115.6
uvicorn[standard]==0.34.0
pydantic==2.10.3
tqdm==4.67.1
requests==2.32.3
huggingface-hub==0.27.0
PyYAML==6.0.2
filelock==3.16.1
```

### Dockerfile

- Base: `python:3.12-slim`
- Install system deps (build-essential, curl, git, libgl1, libglib2.0-0)
- Copy and install requirements FIRST (large layer = biggest SOCI benefit)
- Pre-download model weights so startup is deterministic: `python -c "from transformers import AutoModelForSequenceClassification, AutoTokenizer; AutoTokenizer.from_pretrained('distilbert-base-uncased-finetuned-sst-2-english'); AutoModelForSequenceClassification.from_pretrained('distilbert-base-uncased-finetuned-sst-2-english')"`
- Copy app code LAST (small layer, always pulled immediately)
- EXPOSE 8000
- HEALTHCHECK using curl to /health
- CMD: `uvicorn main:app --host 0.0.0.0 --port 8000 --workers 1`

### Terraform

**Provider:** AWS, use the latest major version constraint (`~> 6.0`), region from variable.

**ECR:** Create repository (configurable via `create_ecr_repository` bool). Set `force_delete = true` for easy cleanup. Enable scan on push.

**ECS Cluster:** Container Insights enabled. Fargate capacity provider.

**Two Task Definitions (no services):**
- `soci-demo-with-soci` — image tag `latest-soci`
- `soci-demo-without-soci` — image tag `latest-no-soci`
- Both: Fargate, awsvpc, runtime platform LINUX/X86_64
- CPU: `4096` (4 vCPU), Memory: `8192` (8 GB) — needed for PyTorch
- Health check: `curl -f http://localhost:8000/health || exit 1`, startPeriod 60s
- Log driver: awslogs, 7-day retention

**No services, no security groups.** Tasks are launched via `aws ecs run-task` in the comparison script. The user provides subnet IDs and security group at run time (not in Terraform).

**Variables must include:**
- `aws_account_id` (string, validated 12 digits)
- `aws_region` (string, default "us-east-1")
- `subnet_ids` (list(string) — at least one, used in run-and-compare script)
- `assign_public_ip` (bool, default false — set true for public subnets without NAT)
- `cluster_name` (string, default "soci-demo")
- `create_ecr_repository` (bool, default true)
- `ecr_repository_name` (string, default "soci-demo-ml-inference")
- `image_tag_soci` / `image_tag_no_soci` (strings)
- `task_cpu` / `task_memory` (strings, defaults "4096" / "8192")
- `task_execution_role_arn` (string, optional — used when user has existing roles)
- `task_role_arn` (string, optional — used when user has existing roles)

**Outputs:** ECR URL, ECR login command (using account ID + region), cluster name, both task definition ARNs, run-task example commands, log group names.

### IAM (terraform/iam.tf)

Generate ONLY if user doesn't have existing roles. Two roles:

1. **Task Execution Role** (`soci-demo-task-execution`):
   - Trust: `ecs-tasks.amazonaws.com`
   - Attach: `arn:aws:iam::aws:policy/service-role/AmazonECSTaskExecutionRolePolicy`

2. **Task Role** (`soci-demo-task`):
   - Trust: `ecs-tasks.amazonaws.com`