ios-testing-patterns

What this skill does

# iOS Testing Patterns Skill

**Comprehensive guide to XCTest and XCUITest execution, analysis, and flaky test detection**

## Overview

This skill provides patterns and workflows for executing iOS tests using XCTest and XCUITest frameworks. It covers test execution strategies, result analysis, flaky test detection, and troubleshooting common test failures.

**Use this skill when:**

- Running unit tests or UI tests
- Analyzing test failures
- Detecting and debugging flaky tests
- Optimizing test execution performance
- Setting up CI/CD test pipelines

## Quick Reference

| Task               | Operation | Key Parameters         |
| ------------------ | --------- | ---------------------- |
| Run all tests      | `test`    | scheme, destination    |
| Run specific test  | `test`    | scheme, only_testing   |
| Skip tests         | `test`    | scheme, skip_testing   |
| Use test plan      | `test`    | scheme, test_plan      |
| Parallel execution | `test`    | destination (multiple) |

## When to Use This Skill

**Use ios-testing-patterns when:**

- Executing XCTest unit tests
- Running XCUITest UI automation tests
- Investigating test failures
- Detecting intermittent test failures (flaky tests)
- Analyzing test performance and execution time
- Setting up test schemes and test plans
- Configuring parallel test execution
- Debugging test-specific simulator issues

**Related Skills:**

- **xcode-workflows**: For general build and test operations
- **simulator-workflows**: For managing test simulators
- **ui-automation-workflows**: For UI test interaction patterns

## Key Concepts

### XCTest vs XCUITest

**XCTest (Unit/Integration Tests):**

- Fast execution (milliseconds to seconds)
- Tests individual components in isolation
- Direct access to app internals
- No UI interaction required
- Runs in same process as app code

**XCUITest (UI Tests):**

- Slower execution (seconds to minutes)
- Tests user-facing workflows
- Black-box testing (no app internals)
- Requires simulator/device UI
- Runs in separate process

### Test Execution Strategies

**Sequential Execution:**

- Tests run one after another
- Predictable, repeatable results
- Slower overall execution
- Better for debugging

**Parallel Execution:**

- Tests run simultaneously on multiple simulators
- Faster overall execution
- Requires more system resources
- May expose race conditions

**Test Sharding:**

- Split test suite across multiple destinations
- Optimal for CI/CD pipelines
- Reduces total execution time
- Requires careful test isolation

## Workflows

### 1. Running Unit Tests

**Basic Unit Test Execution:**

```json
{
  "operation": "test",
  "scheme": "MyApp",
  "destination": "platform=iOS Simulator,name=iPhone 15,OS=18.0"
}
```

**Note:** The destination parameter now supports auto-resolution! You can omit the OS version (e.g., `"platform=iOS Simulator,name=iPhone 15"`) and the tool will automatically select the latest available OS version for that device.

**Run Specific Test Class:**

```json
{
  "operation": "test",
  "scheme": "MyApp",
  "destination": "platform=iOS Simulator,name=iPhone 15,OS=18.0",
  "options": {
    "only_testing": ["MyAppTests/NetworkTests"]
  }
}
```

**Run Specific Test Method:**

```json
{
  "operation": "test",
  "scheme": "MyApp",
  "destination": "platform=iOS Simulator,name=iPhone 15,OS=18.0",
  "options": {
    "only_testing": ["MyAppTests/NetworkTests/testAPIRequest"]
  }
}
```

### 2. Running UI Tests

**Basic UI Test Execution:**

```json
{
  "operation": "test",
  "scheme": "MyAppUITests",
  "destination": "platform=iOS Simulator,name=iPhone 15"
}
```

**UI Tests with Specific Device:**

```json
{
  "operation": "test",
  "scheme": "MyAppUITests",
  "destination": "platform=iOS Simulator,name=iPad Pro (12.9-inch)"
}
```

**UI Tests with Test Plan:**

```json
{
  "operation": "test",
  "scheme": "MyApp",
  "destination": "platform=iOS Simulator,name=iPhone 15",
  "options": {
    "test_plan": "SmokeTests"
  }
}
```

### 3. Parallel Test Execution

**Multiple Destinations:**

Run tests on multiple simulators simultaneously:

```json
{
  "operation": "test",
  "scheme": "MyApp",
  "destination": [
    "platform=iOS Simulator,name=iPhone 15",
    "platform=iOS Simulator,name=iPhone SE (3rd generation)"
  ],
  "options": {
    "parallel": true
  }
}
```

**Parallel Test Benefits:**

- Reduces total execution time
- Tests across multiple device types
- Exposes device-specific issues
- Better CI/CD performance

**Parallel Test Considerations:**

- Requires multiple simulators
- Higher CPU/memory usage
- May expose race conditions
- Test isolation critical

### 4. Test Filtering

**Skip Specific Tests:**

```json
{
  "operation": "test",
  "scheme": "MyApp",
  "destination": "platform=iOS Simulator,name=iPhone 15",
  "options": {
    "skip_testing": [
      "MyAppUITests/SlowTests",
      "MyAppTests/NetworkTests/testLargeDownload"
    ]
  }
}
```

**Run Only Fast Tests:**

```json
{
  "operation": "test",
  "scheme": "MyApp",
  "destination": "platform=iOS Simulator,name=iPhone 15",
  "options": {
    "only_testing": ["MyAppTests/UnitTests"],
    "skip_testing": ["MyAppTests/UnitTests/testSlowOperation"]
  }
}
```

### 5. Test Result Analysis

**Success Response:**

```json
{
  "success": true,
  "tests_run": 45,
  "tests_passed": 45,
  "tests_failed": 0,
  "execution_time": "12.4s",
  "scheme": "MyApp"
}
```

**Failure Response:**

```json
{
  "success": false,
  "tests_run": 45,
  "tests_passed": 43,
  "tests_failed": 2,
  "failures": [
    {
      "test": "MyAppTests.LoginTests.testInvalidPassword",
      "message": "XCTAssertEqual failed: (\"error\") is not equal to (\"invalid\")",
      "file": "LoginTests.swift",
      "line": 42
    },
    {
      "test": "MyAppUITests.CheckoutTests.testPaymentFlow",
      "message": "Failed to find button \"Confirm\"",
      "file": "CheckoutTests.swift",
      "line": 78
    }
  ],
  "execution_time": "18.7s"
}
```

**Analyzing Test Results:**

1. Check `success` status
2. Review `tests_failed` count
3. Examine `failures` array for details
4. Check `execution_time` for performance issues
5. Look for patterns in failure messages

### 6. Flaky Test Detection

**What are Flaky Tests?**

Tests that intermittently pass or fail without code changes:

- Timing-dependent assertions
- Race conditions
- Async operation issues
- Shared state between tests
- Network-dependent tests
- UI animation timing issues

**Detection Strategy 1: Multiple Runs**

Run tests multiple times to detect flakiness:

```bash
# Run tests 5 times and compare results
for i in {1..5}; do
  execute_xcode_command(test, scheme, destination)
  # Record results
done
```

**Detection Strategy 2: Pattern Analysis**

Look for common flaky test patterns:

```json
{
  "operation": "test",
  "scheme": "MyApp",
  "destination": "platform=iOS Simulator,name=iPhone 15",
  "options": {
    "only_testing": ["MyAppTests/SuspectedFlakyTest"]
  }
}
```

**Flaky Test Indicators:**

1. **Timeout failures**: "Exceeded timeout of 30 seconds"
2. **Element not found**: "Failed to find button..." (UI tests)
3. **Race conditions**: "Expected 5 but got 4"
4. **Async timing**: "Asynchronous wait failed"
5. **Network failures**: "Request timed out"

**Flaky Test Workflow:**

```
1. Identify suspect test (intermittent failures)
2. Run test 10+ times in isolation
3. Analyze failure patterns
4. Check for:
   - Hard-coded waits (sleep/wait)
   - Expectation timeouts too short
   - Shared mutable state
   - Network dependencies
   - Animation timing assumptions
5. Fix root cause
6. Verify with repeated runs
```

### 7. Test Isolation

**Clean State Before Tests:**

```json
{
  "operation": "test",
  "scheme": "MyApp",
  "destination": "platform=iOS Simulator,name=iPhone 15",
  "options": {
    "clean_before_build": true
  }
}
```

**Reset Simulator State:**

```bash
# Use simulator-workflows skill
execute_simulator_command(operation: "erase", udid: <device-udid>)
```

**Test I