regression-auto-baseline

What this skill does

# regression-auto-baseline

Automatically manage regression test baseline lifecycle with triggers for releases, deployments, and quality gates.

## Triggers


Alternate expressions and non-obvious activations (primary phrases are matched automatically from the skill description):

- "auto-update baselines" → automatic baseline management
- "baseline on release" → release-triggered baseline update

## Purpose

This skill automates regression baseline management by:
- Triggering baseline updates on semantic version tags
- Updating baselines after successful merges to main/master
- Creating baselines on deployment success
- Scheduling periodic baseline refreshes
- Managing baseline retention and archival
- Validating baseline quality before activation
- Integrating with CI/CD for seamless automation

## Behavior

When triggered or activated automatically, this skill:

1. **Detects trigger conditions**:
   - Semantic version tag created (v*.*.*)
   - Merge to protected branch completed
   - Deployment succeeded to environment
   - Quality gate passed
   - Manual approval received
   - Scheduled time reached

2. **Validates pre-conditions**:
   - Tests are passing (minimum threshold met)
   - No critical issues open
   - Quality gates passed
   - Human approval obtained (if required)
   - No concurrent baseline updates in progress

3. **Captures new baseline**:
   - Run full regression test suite
   - Capture all artifact types (functional, visual, performance, API)
   - Verify baseline stability (run multiple times if configured)
   - Generate checksums for integrity
   - Tag with git commit, release version, timestamp

4. **Validates baseline quality**:
   - Compare to previous baseline
   - Detect unexpected changes
   - Verify all expected outputs present
   - Check for baseline drift
   - Flag suspicious differences for review

5. **Activates baseline**:
   - Archive previous baseline version
   - Set new baseline as active
   - Update baseline manifest
   - Notify relevant teams
   - Create baseline comparison report

6. **Manages lifecycle**:
   - Archive old baselines per retention policy
   - Compress large baselines
   - Upload to cloud storage if configured
   - Prune outdated baselines
   - Maintain audit trail

## Trigger Types

### Semantic Version Tag

```yaml
trigger:
  type: git_tag
  pattern: "^v[0-9]+\\.[0-9]+\\.[0-9]+$"
  description: "Baseline on semantic version tags"

  behavior:
    on_tag_create:
      - validate_tag_format
      - check_if_tests_passing
      - create_baseline:
          name: "release-{tag}"
          auto_activate: true
      - tag_baseline_with_git_tag

  example:
    tag: v2.1.0
    baseline_id: release-v2.1.0
    status: auto-created
```

### Merge to Main

```yaml
trigger:
  type: git_merge
  branches:
    - main
    - master
  description: "Baseline after merge to protected branch"

  behavior:
    on_merge_complete:
      - wait_for_ci_success
      - check_test_pass_rate:
          minimum: 95%
      - create_baseline:
          name: "main-{commit_short}"
          auto_activate: false  # Require approval
      - request_approval:
          approvers: [tech-lead, test-lead]

  example:
    merge_commit: abc123def
    baseline_id: main-abc123d
    approval_required: true
```

### Deployment Success

```yaml
trigger:
  type: deployment
  environments:
    - production
    - staging
  description: "Baseline after successful deployment"

  behavior:
    on_deployment_success:
      - wait_for_smoke_tests
      - verify_health_checks
      - create_baseline:
          name: "{environment}-{deploy_id}"
          auto_activate: true
      - link_to_deployment:
          id: deploy_id
          timestamp: deployment_timestamp

  example:
    environment: production
    deploy_id: prod-2026-01-28-01
    baseline_id: production-prod-2026-01-28-01
```

### Manual Approval

```yaml
trigger:
  type: manual
  description: "Baseline after explicit human approval"

  behavior:
    on_approval_received:
      - validate_approver_permissions
      - capture_approval_metadata
      - create_baseline:
          name: "approved-{timestamp}"
          auto_activate: true
      - record_approver:
          name: approver_name
          reason: approval_reason

  example:
    approver: tech-lead
    reason: "Post-release validation complete"
    baseline_id: approved-2026-01-28T15-30
```

### Scheduled Updates

```yaml
trigger:
  type: schedule
  description: "Baseline on fixed schedule"

  schedules:
    nightly:
      cron: "0 2 * * *"
      description: "Nightly baseline capture"
      auto_activate: false

    weekly:
      cron: "0 3 * * 0"
      description: "Weekly baseline for long-term tracking"
      auto_activate: false

    monthly:
      cron: "0 4 1 * *"
      description: "Monthly baseline archive"
      auto_activate: false

  behavior:
    on_schedule_trigger:
      - check_if_tests_healthy
      - create_baseline:
          name: "{schedule}-{date}"
          auto_activate: false
      - generate_drift_report
```

## Baseline Storage Strategies

### Local File Storage

```yaml
storage:
  type: local
  location: .aiwg/testing/baselines/
  structure:
    - functional/
    - visual/
    - performance/
    - api/

  retention:
    active: 1        # Keep 1 active baseline per type
    archive: 10      # Keep 10 archived versions
    compress: true   # Compress archives with gzip

  cleanup:
    prune_after_days: 90
    keep_tagged: true  # Never prune tagged baselines
```

### Git LFS

```yaml
storage:
  type: git_lfs
  description: "Use Git LFS for large baselines (screenshots, large JSON)"

  configuration:
    track_patterns:
      - "*.png"
      - "*.jpg"
      - "baselines/**/*.json"
    max_file_size: 10MB

  behavior:
    on_baseline_create:
      - compress_if_needed
      - add_to_git_lfs
      - commit_with_metadata
```

### Cloud Storage (S3/GCS)

```yaml
storage:
  type: cloud
  provider: s3  # or gcs, azure_blob

  configuration:
    bucket: my-project-baselines
    path_prefix: baselines/{environment}/
    encryption: AES256

  behavior:
    on_baseline_create:
      - upload_to_cloud
      - generate_signed_url:
          expiry: 7 days
      - store_url_in_manifest
      - keep_local_copy: false  # Optional: delete after upload

  retention:
    lifecycle_policy:
      - after_30_days: transition_to_infrequent_access
      - after_90_days: transition_to_glacier
      - after_365_days: delete
```

### Artifact Registry

```yaml
storage:
  type: artifact_registry
  description: "Integrate with artifact registries (npm, Maven, Docker)"

  configuration:
    registry_url: https://registry.example.com
    repository: baseline-artifacts
    versioning: semver  # Follow package versioning

  behavior:
    on_baseline_create:
      - package_baseline:
          format: tarball
          name: "{project}-baseline"
          version: "{tag_version}"
      - publish_to_registry
      - tag_as_latest: true
```

## Baseline Validation

### Pre-Activation Checks

```yaml
validation:
  before_activation:
    - test_pass_rate:
        minimum: 95%
        description: "At least 95% of tests must pass"

    - quality_gates:
        all_must_pass: true
        description: "All quality gates must be green"

    - stability_check:
        runs: 3
        consistency: 100%
        description: "Baseline must be 100% consistent across 3 runs"

    - critical_issues:
        max_open: 0
        description: "No critical issues can be open"

    - human_approval:
        required_for:
          - production_baselines
          - breaking_changes
        approvers: [tech-lead, qa-lead]
```

### Baseline Diff Detection

```yaml
validation:
  diff_detection:
    compare_to_previous: true
    flag_changes:
      - new_failures
      - performance_degradation
      - unexpected_schema_changes
      - visual_differences

    severity_classification:
      critical:
        - new_test_failures
        - security_regression