microservices-patterns

What this skill does

# Microservices Patterns

A comprehensive skill for building, deploying, and managing production-grade microservices architectures. This skill covers service mesh patterns, traffic management, resilience engineering, observability, security, and modern microservices best practices using Istio and Kubernetes.

## When to Use This Skill

Use this skill when:

- Architecting microservices-based applications with distributed systems
- Implementing service mesh infrastructure for service-to-service communication
- Adding resilience patterns like circuit breakers, retries, and timeouts
- Managing traffic routing, load balancing, and canary deployments
- Implementing distributed tracing and observability across microservices
- Securing microservices with mTLS and authorization policies
- Troubleshooting cascading failures and service degradation
- Building fault-tolerant distributed systems
- Implementing blue-green deployments and A/B testing
- Managing multi-cluster microservices deployments
- Implementing chaos engineering and fault injection
- Migrating from monolithic to microservices architecture

## Core Concepts

### Microservices Architecture

Microservices architecture structures an application as a collection of loosely coupled services:

- **Service Independence**: Each service is independently deployable and scalable
- **Domain-Driven Design**: Services align with business capabilities
- **Decentralized Data**: Each service owns its data store
- **API-First**: Services communicate via well-defined APIs
- **Polyglot Persistence**: Different services can use different databases
- **Failure Isolation**: Service failures don't cascade across the system

### Service Mesh Fundamentals

A service mesh is an infrastructure layer for handling service-to-service communication:

- **Data Plane**: Sidecar proxies (Envoy) deployed alongside each service
- **Control Plane**: Manages and configures proxies (Istio, Linkerd, Consul)
- **Service Discovery**: Automatic service registration and discovery
- **Load Balancing**: Intelligent traffic distribution across service instances
- **Observability**: Built-in metrics, logs, and distributed tracing
- **Security**: mTLS, authentication, and authorization

### Istio Architecture

Istio is the most popular service mesh implementation:

**Control Plane Components:**
- **Istiod**: Unified control plane for service discovery, configuration, and certificate management
- **Pilot**: Traffic management and service discovery
- **Citadel**: Certificate authority for mTLS
- **Galley**: Configuration validation and distribution

**Data Plane:**
- **Envoy Proxy**: High-performance sidecar proxy for each service
- **Iptables Rules**: Transparent traffic interception
- **Service Proxy**: Handles all network traffic for the service

### Key Service Mesh Patterns

1. **Sidecar Pattern**: Proxy deployed alongside application container
2. **Service Discovery**: Automatic registration and discovery of services
3. **Traffic Splitting**: Route percentage of traffic to different versions
4. **Circuit Breaker**: Prevent cascading failures
5. **Retry Logic**: Automatic retry with exponential backoff
6. **Timeout Policies**: Request timeout configuration
7. **Fault Injection**: Chaos testing in production
8. **Rate Limiting**: Protect services from overload
9. **mTLS**: Mutual TLS for service-to-service encryption
10. **Distributed Tracing**: Request flow across services

## Traffic Management

### Virtual Services

Virtual services define routing rules for traffic within the mesh:

**Key Features:**
- **HTTP/TCP/TLS Routing**: Protocol-specific routing rules
- **Match Conditions**: Route based on headers, URIs, methods
- **Weighted Routing**: Traffic splitting across versions
- **Redirects and Rewrites**: URL manipulation
- **Fault Injection**: Delay and abort injection
- **Retries**: Automatic retry configuration
- **Timeouts**: Request timeout policies

**Virtual Service Structure:**
```yaml
apiVersion: networking.istio.io/v1beta1
kind: VirtualService
metadata:
  name: reviews
spec:
  hosts:
  - reviews
  http:
  - match:
    - headers:
        end-user:
          exact: jason
    route:
    - destination:
        host: reviews
        subset: v2
  - route:
    - destination:
        host: reviews
        subset: v3
```

### Destination Rules

Destination rules configure policies for traffic after routing:

**Key Features:**
- **Load Balancing**: Round robin, random, least request
- **Connection Pools**: Connection limits and timeouts
- **Outlier Detection**: Circuit breaker configuration
- **TLS Settings**: mTLS mode configuration
- **Subset Definitions**: Version-based service subsets

**Common Load Balancing Strategies:**
- **ROUND_ROBIN**: Default, distributes evenly
- **LEAST_REQUEST**: Routes to instances with fewest requests
- **RANDOM**: Random distribution
- **PASSTHROUGH**: Use original destination

### Traffic Splitting

Traffic splitting enables gradual rollouts and A/B testing:

**Use Cases:**
- **Canary Deployments**: Route small percentage to new version
- **Blue-Green Deployments**: Switch traffic between versions
- **A/B Testing**: Split traffic for experimentation
- **Dark Launches**: Shadow traffic to new version

**Progressive Delivery Pattern:**
```
v1: 100% → 90% → 70% → 50% → 20% → 0%
v2:   0% → 10% → 30% → 50% → 80% → 100%
```

### Gateway Configuration

Gateways manage ingress and egress traffic:

**Ingress Gateway:**
- External traffic entry point
- TLS termination
- Protocol-specific routing
- Virtual hosting

**Egress Gateway:**
- Control outbound traffic
- Security policies for external services
- Traffic monitoring and logging

## Resilience Patterns

### Circuit Breaker Pattern

Circuit breakers prevent cascading failures by detecting and isolating failing services:

**States:**
- **Closed**: Normal operation, requests flow through
- **Open**: Service failing, requests fail immediately
- **Half-Open**: Testing if service recovered

**Configuration Parameters:**
- **Consecutive Errors**: Errors before opening circuit
- **Interval**: Time window for error counting
- **Base Ejection Time**: How long to eject failing instances
- **Max Ejection Percentage**: Maximum percentage of pool to eject

**Benefits:**
- Prevents resource exhaustion
- Fails fast instead of waiting for timeouts
- Gives failing services time to recover
- Monitors service health automatically

### Retry Logic

Automatic retry with intelligent backoff strategies:

**Retry Strategies:**
- **Fixed Delay**: Constant delay between retries
- **Exponential Backoff**: Increasing delay between retries
- **Jittered Backoff**: Random jitter to prevent thundering herd

**Configuration:**
- **Attempts**: Maximum number of retries
- **Per Try Timeout**: Timeout for each attempt
- **Retry On**: Conditions triggering retry (5xx, timeout, refused-stream)
- **Backoff**: Base interval and maximum interval

**Best Practices:**
- Only retry idempotent operations
- Use exponential backoff with jitter
- Set maximum retry attempts
- Monitor retry rates

### Timeout Policies

Timeout policies prevent indefinite waiting:

**Timeout Types:**
- **Request Timeout**: End-to-end request timeout
- **Per Try Timeout**: Timeout for each retry attempt
- **Idle Timeout**: Connection idle timeout
- **Connection Timeout**: Initial connection timeout

**Timeout Hierarchy:**
```
Overall Request Timeout
├─ Retry 1 (Per Try Timeout)
├─ Retry 2 (Per Try Timeout)
└─ Retry 3 (Per Try Timeout)
```

**Best Practices:**
- Set timeouts based on SLA requirements
- Use shorter timeouts for critical paths
- Configure per-try timeouts lower than overall timeout
- Monitor timeout rates and adjust

### Bulkhead Pattern

Bulkheads isolate resources to prevent complete system failure:

**Implementation:**
- **Thread Pools**: Separate thread pools per service
- **Connection Pools**: Limited connections per upstream
- **Queue Limits**: Bounded queues to prevent memory issues
- **Semaphores**: Lim