testing-strategies

What this skill does

# Testing Strategies

Build comprehensive, effective test suites by strategically selecting and implementing the right testing approaches across unit, integration, E2E, and contract testing levels.

## Purpose

This skill provides strategic frameworks for:
- **Test Type Selection**: Determine when to use unit vs. integration vs. E2E vs. contract testing
- **Test Pyramid Balancing**: Optimize test distribution for fast feedback and reliable coverage
- **Multi-Language Implementation**: Apply consistent testing patterns across TypeScript, Python, Go, and Rust
- **Test Data Management**: Choose appropriate strategies (fixtures, factories, property-based testing)
- **CI/CD Integration**: Integrate tests into automated pipelines with optimal execution patterns

Testing is foundational to reliable software. With microservices architectures and continuous delivery becoming standard in 2025, strategic testing across multiple levels is more critical than ever.

## When to Use This Skill

Invoke this skill when:
- Building a new feature that requires test coverage
- Designing a testing strategy for a new project
- Refactoring existing tests to improve speed or reliability
- Setting up CI/CD pipelines with testing stages
- Choosing between unit, integration, or E2E testing approaches
- Implementing contract testing for microservices
- Managing test data with fixtures, factories, or property-based testing

## The Testing Pyramid Framework

### Core Concept

The testing pyramid guides test distribution for optimal speed and confidence:

```
         /\
        /  \  E2E Tests (10%)
       /----\  - Slow but comprehensive
      /      \  - Full stack validation
     /--------\
    /          \  Integration Tests (20-30%)
   /            \  - Moderate speed
  /--------------\  - Component interactions
 /                \
/------------------\  Unit Tests (60-70%)
                      - Fast feedback
                      - Isolated units
```

**Key Principle**: More unit tests (fast, isolated), fewer E2E tests (slow, comprehensive). Integration tests bridge the gap.

### Modern Adaptations (2025)

**Microservices Adjustment**:
- Add contract testing layer between unit and integration
- Increase integration/contract tests to 30% (validate service boundaries)
- Reduce E2E tests to critical user journeys only

**Cloud-Native Patterns**:
- Use containers for integration tests (ephemeral databases, test services)
- Parallel execution for fast CI/CD feedback
- Risk-based test prioritization (focus on high-impact areas)

For detailed pyramid guidance, see `references/testing-pyramid.md`.

## Universal Testing Decision Tree

### Which Test Type Should I Use?

```
START: Need to test [feature]

Q1: Does this involve multiple systems/services?
  ├─ YES → Q2
  └─ NO  → Q3

Q2: Is this a critical user-facing workflow?
  ├─ YES → E2E Test (complete user journey)
  └─ NO  → Integration or Contract Test

Q3: Does this interact with external dependencies (DB, API, filesystem)?
  ├─ YES → Integration Test (real DB, mocked API)
  └─ NO  → Q4

Q4: Is this pure business logic or a pure function?
  ├─ YES → Unit Test (fast, isolated)
  └─ NO  → Component or Integration Test
```

### Test Type Selection Examples

| Feature | Test Type | Rationale |
|---------|-----------|-----------|
| `calculateTotal(items)` | Unit | Pure function, no dependencies |
| `POST /api/users` endpoint | Integration | Tests API + database interaction |
| User registration flow (form → API → redirect) | E2E | Critical user journey, full stack |
| Microservice A → B communication | Contract | Service interface validation |
| `formatCurrency(amount, locale)` | Unit + Property | Pure logic, many edge cases |
| Form validation logic | Unit | Isolated business rules |
| File upload to S3 | Integration | External service interaction |

For comprehensive decision frameworks, see `references/decision-tree.md`.

## Testing Levels in Detail

### Unit Testing (Foundation - 60-70%)

**Purpose**: Validate small, isolated units of code (functions, methods, components)

**Characteristics**:
- Fast (milliseconds per test)
- Isolated (no external dependencies)
- Deterministic (same input = same output)
- Broad coverage (many tests, small scope each)

**When to Use**:
- Pure functions (input → output)
- Business logic and algorithms
- Utility functions
- Component rendering (without integration)
- Validation logic

**Recommended Tools**:
- **TypeScript/JavaScript**: Vitest (primary, 10x faster than Jest), Jest (legacy)
- **Python**: pytest (industry standard)
- **Go**: testing package (stdlib) + testify (assertions)
- **Rust**: cargo test (stdlib)

For detailed patterns, see `references/unit-testing-patterns.md`.

### Integration Testing (Middle Layer - 20-30%)

**Purpose**: Validate interactions between components, modules, or services

**Characteristics**:
- Moderate speed (seconds per test)
- Partial integration (real database, mocked external APIs)
- Focused scope (test component boundaries)
- API and database validation

**When to Use**:
- API endpoints (request → response)
- Database operations (CRUD, queries)
- Service-to-service communication
- Event handlers and message processing
- File I/O operations

**Recommended Tools**:
- **TypeScript/JavaScript**: Vitest + MSW (API mocking), Supertest (HTTP testing)
- **Python**: pytest + pytest-httpserver, pytest-postgresql
- **Go**: testing + httptest, testcontainers
- **Rust**: cargo test + mockito, testcontainers

For detailed patterns, see `references/integration-testing-patterns.md`.

### End-to-End Testing (Top Layer - 10%)

**Purpose**: Validate complete user workflows across the entire application stack

**Characteristics**:
- Slow (minutes per test suite)
- Full integration (real browser, services, database)
- Wide scope (user journeys from start to finish)
- Prone to flakiness (requires careful design)

**When to Use**:
- Critical user journeys (login, checkout, payment)
- Cross-browser compatibility validation
- Real-world scenarios not covered by integration tests
- Regression prevention for core features

**Best Practices**:
- Limit E2E tests to high-value scenarios (not every edge case)
- Use stable selectors (data-testid, not CSS classes)
- Implement retry logic for network flakiness
- Run tests in parallel for speed

**Recommended Tools**:
- **All Languages**: Playwright (cross-browser, fast, Microsoft-backed)

For detailed patterns, see `references/e2e-testing-patterns.md`.

### Contract Testing (Microservices)

**Purpose**: Validate service interfaces and API contracts without full integration

**When to Use**:
- Microservices architecture
- Service-to-service communication
- API contract validation
- Reducing E2E testing overhead

**Recommended Tool**: Pact (pact.io) - supports TypeScript, Python, Go, Rust

For detailed patterns, see `references/contract-testing.md`.

## Test Data Management Strategies

### When to Use Each Approach

**Fixtures (Static Data)**:
- Pros: Deterministic, easy to debug
- Cons: Can become stale, doesn't test variety
- Use When: Testing known scenarios, regression tests

**Factories (Generated Data)**:
- Pros: Flexible, generates variety
- Cons: Less deterministic, harder to debug
- Use When: Need diverse test data, testing edge cases

**Property-Based Testing (Random Data)**:
- Pros: Finds edge cases not anticipated
- Cons: Can be slow, failures harder to reproduce
- Use When: Complex algorithms, parsers, validators

**Recommended Combination**:
- **Unit Tests**: Fixtures (known inputs) + Property-Based (edge cases)
- **Integration Tests**: Factories (flexible data) + Database seeding
- **E2E Tests**: Fixtures (reproducible scenarios)

**Property-Based Testing Tools**:
- **TypeScript/JavaScript**: fast-check
- **Python**: hypothesis (best-in-class)
- **Go**: gopter
- **Rust**: proptest (primary)

For detailed strategies, see `references/test-data-strategies.md`.

## Mocking Decision Matrix

### When to Mock vs. Use Real Dependenc