software-backend

What this skill does

# Software Backend Engineering

Use this skill to design, implement, and review production-grade backend services: API boundaries, data layer, auth, caching, observability, error handling, testing, and deployment.

Defaults to bias toward: type-safe boundaries (validation at the edge), OpenTelemetry for observability, zero-trust assumptions, idempotency for retries, RFC 9457 errors, Postgres + pooling, structured logs, timeouts, and rate limiting.

**Scaffolding rule**: When scaffolding a new project, show full working implementations for all domain logic — fraud rules, audit logging, webhook handlers, validation pipelines, background jobs. Don't just reference file names or stub functions; show the actual code so the user can run it immediately.

---

## Quick Reference

| Task | Default Picks | Notes |
|------|---------------|-------|
| REST API | Fastify / Express / NestJS | Prefer typed boundaries + explicit timeouts |
| Edge API | Hono / platform-native handlers | Keep work stateless, CPU-light |
| Type-Safe API | tRPC | Prefer for TS monorepos and internal APIs |
| GraphQL API | Apollo Server / Pothos | Prefer for complex client-driven queries |
| Database | PostgreSQL | Use pooling + migrations + query budgets |
| ORM / Query Layer | Prisma / Drizzle / SQLAlchemy / GORM / SeaORM / EF Core | Prefer explicit transactions |
| Authentication | OIDC/OAuth + sessions/JWT | Prefer httpOnly cookies for browsers |
| Validation | Zod / Pydantic / validator libs | Validate at the boundary, not deep inside |
| Caching | Redis (or managed) | Use TTLs + invalidation strategy |
| Background Jobs | BullMQ / platform queues | Make jobs idempotent + retry-safe |
| Testing | Unit + integration + contract/E2E | Keep most tests below the UI layer |
| Observability | Structured logs + OpenTelemetry | Correlation IDs end-to-end |

## Scope

Use this skill to:

- Design and implement REST/GraphQL/tRPC APIs
- Model data schemas and run safe migrations
- Implement authentication/authorization (OIDC/OAuth, sessions/JWT)
- Add validation, error handling, rate limiting, caching, and background jobs
- Ship production readiness (timeouts, observability, deploy/runbooks)

## When NOT to Use This Skill

Use a different skill when:

- **Frontend-only concerns** -> See [software-frontend](../software-frontend/SKILL.md)
- **Infrastructure provisioning (Terraform, K8s manifests)** -> See [ops-devops-platform](../ops-devops-platform/SKILL.md)
- **API design patterns only (no implementation)** -> See [dev-api-design](../dev-api-design/SKILL.md)
- **SQL query optimization and indexing** -> See [data-sql-optimization](../data-sql-optimization/SKILL.md)
- **Security audits and threat modeling** -> See [software-security-appsec](../software-security-appsec/SKILL.md)
- **System architecture (beyond single service)** -> See [software-architecture-design](../software-architecture-design/SKILL.md)

## Technology Selection

Pick based on the strongest constraint, not feature lists:

| Constraint | Default Pick | Why |
|-----------|-------------|-----|
| Team knows TypeScript only | Fastify/Hono + Prisma/Drizzle | Ecosystem depth, hiring ease |
| Need <50ms P95, CPU-bound work | Go (net/http + sqlc/pgx) | Goroutines isolate CPU work; no event-loop risk |
| Data-heavy / ML integration | Python (FastAPI + SQLAlchemy) | Best ecosystem for numpy/pandas/ML pipelines |
| Memory-safety critical | Rust (Axum + SeaORM/SQLx) | Zero-cost abstractions, no GC |
| Enterprise/.NET team | C# (ASP.NET Core + EF Core) | Azure integration, mature tooling |
| Edge/serverless | Hono / platform-native handlers | Stateless, CPU-light, fast cold starts |
| Fintech/audit-sensitive | Go + sqlc (or raw SQL) | ORM magic is a liability; you need auditable SQL |

For detailed framework/ORM/auth/caching selection trees, see [references/edge-deployment-guide.md](references/edge-deployment-guide.md) and language-specific references.
See [assets/](assets/) for starter templates per language.

---

## API Design Patterns (Dec 2025)

### Idempotency Patterns

All mutating operations MUST support idempotency for retry safety.

**Implementation:**

```typescript
// Idempotency key header
const idempotencyKey = request.headers['idempotency-key'];
const cached = await redis.get(`idem:${idempotencyKey}`);
if (cached) return JSON.parse(cached);

const result = await processOperation();
await redis.set(`idem:${idempotencyKey}`, JSON.stringify(result), 'EX', 86400);
return result;
```

| Do | Avoid |
|----|-------|
| Store idempotency keys with TTL (24h typical) | Processing duplicate requests |
| Return cached response for duplicate keys | Different responses for same key |
| Use client-generated UUIDs | Server-generated keys |

### Pagination Patterns

| Pattern | Use When | Example |
|---------|----------|---------|
| Cursor-based | Large datasets, real-time data | `?cursor=abc123&limit=20` |
| Offset-based | Small datasets, random access | `?page=3&per_page=20` |
| Keyset | Sorted data, high performance | `?after_id=1000&limit=20` |

**Prefer cursor-based pagination** for APIs with frequent inserts.

### Error Response Standard (Problem Details)

Use a consistent machine-readable error format (RFC 9457 Problem Details): https://www.rfc-editor.org/rfc/rfc9457

```json
{
  "type": "https://example.com/problems/invalid-request",
  "title": "Invalid request",
  "status": 400,
  "detail": "email is required",
  "instance": "/v1/users"
}
```

### Health Check Patterns

```typescript
// Liveness: Is the process running?
app.get('/health/live', (req, res) => {
  res.status(200).json({ status: 'ok' });
});

// Readiness: Can the service handle traffic?
app.get('/health/ready', async (req, res) => {
  const dbOk = await checkDatabase();
  const cacheOk = await checkRedis();
  if (dbOk && cacheOk) {
    res.status(200).json({ status: 'ready', db: 'ok', cache: 'ok' });
  } else {
    res.status(503).json({ status: 'not ready', db: dbOk, cache: cacheOk });
  }
});
```

### Common Mistakes (Non-Obvious)

| Avoid | Instead | Why |
|-------|---------|-----|
| N+1 queries | `include`/`select` or DataLoader | 10-100x perf hit; easy to miss in ORM code |
| No request timeouts | Timeouts on HTTP clients, DB, handlers | Hung deps cascade; see Production Hardening below |
| Missing connection pooling | Prisma pool / PgBouncer / pgx pool | Exhaustion under load on shared DB tiers |
| Catching errors silently | Log + rethrow or handle explicitly | Hidden failures, impossible to debug |

---

## Production Hardening: Patterns Models Skip

These are the patterns that separate "works in dev" from "survives production." Models tend to skip them unless explicitly prompted — add them to every service.

### Request & Query Timeouts

Every outbound call needs a timeout. Without one, a hung dependency leaks connections and cascades failures.

```typescript
// HTTP client timeout
const response = await fetch(url, { signal: AbortSignal.timeout(5000) });

// Database query timeout (Prisma)
await prisma.$queryRaw`SET statement_timeout = '3000'`;

// Express/Fastify request timeout
server.register(import('@fastify/timeout'), { timeout: 30000 });
```

| Layer | Default Timeout | Rationale |
|-------|----------------|-----------|
| HTTP client calls | 5s | External APIs shouldn't block you |
| Database queries | 3s | Slow queries = missing index or bad plan |
| Request handler | 30s | Safety net for the whole request lifecycle |
| Background jobs | 5min | Jobs that run longer need chunking |

### Field-Level Selection (Don't `SELECT *`)

ORMs default to fetching all columns. On wide tables this wastes bandwidth and hides performance problems.

```typescript
// BAD: fetches all 30 columns
const users = await prisma.user.findMany({ include: { posts: true } });

// GOOD: fetch only what the endpoint needs
const users = await prisma.user.findMany({
  select: { id: true, name: true, email: true },
  include: { posts: { select: { id: true, title: true } } }
});
```

For Go