e2e-test-scenarios

What this skill does

# e2e-test-scenarios

## Instructions

This skill provides comprehensive end-to-end testing capabilities for Supabase applications, covering database operations, authentication flows, AI features (pgvector), realtime subscriptions, and production readiness validation.

### Phase 1: Setup Test Environment

1. Initialize test environment:
   ```bash
   bash scripts/setup-test-env.sh
   ```
   This creates:
   - Test database configuration
   - Environment variables for testing
   - Test data fixtures
   - CI/CD configuration templates

2. Configure test database:
   - Use dedicated test project or local Supabase instance
   - Never run tests against production
   - Set `SUPABASE_TEST_URL` and `SUPABASE_TEST_ANON_KEY`
   - Enable pgTAP extension for database tests

3. Install dependencies:
   ```bash
   npm install --save-dev @supabase/supabase-js jest @types/jest
   # or
   pnpm add -D @supabase/supabase-js vitest
   ```

### Phase 2: Database Workflow Testing

Test complete database operations from schema to queries:

1. Run database E2E tests:
   ```bash
   bash scripts/test-database-workflow.sh
   ```

   This validates:
   - Schema creation and migrations
   - Table relationships and foreign keys
   - RLS policies enforcement
   - Triggers and functions
   - Data integrity constraints

2. Use pgTAP for SQL-level tests:
   ```bash
   # Run all database tests
   supabase test db

   # Run specific test file
   supabase test db --file tests/database/users.test.sql
   ```

3. Test schema migrations:
   ```bash
   # Test migration up/down
   supabase db reset --linked
   supabase db push

   # Verify schema state
   bash scripts/validate-schema.sh
   ```

### Phase 3: Authentication Flow Testing

Test complete auth workflows end-to-end:

1. Run authentication E2E tests:
   ```bash
   bash scripts/test-auth-workflow.sh
   ```

   This validates:
   - Email/password signup and login
   - Magic link authentication
   - OAuth provider flows (Google, GitHub, etc.)
   - Session management and refresh
   - Password reset flows
   - Email verification
   - Multi-factor authentication (MFA)

2. Test RLS with authenticated users:
   ```typescript
   // See templates/auth-tests.ts for complete examples
   test('authenticated users see only their data', async () => {
     const { data, error } = await supabase
       .from('private_notes')
       .select('*');

     expect(data).toHaveLength(userNoteCount);
     expect(data.every(note => note.user_id === userId)).toBe(true);
   });
   ```

3. Test session persistence:
   ```bash
   # Run session validation tests
   npm test -- auth-session.test.ts
   ```

### Phase 4: AI Features Testing (pgvector)

Test vector search and semantic operations:

1. Run AI features E2E tests:
   ```bash
   bash scripts/test-ai-features.sh
   ```

   This validates:
   - pgvector extension is enabled
   - Embedding storage and retrieval
   - Vector similarity search accuracy
   - HNSW/IVFFlat index performance
   - Hybrid search (keyword + semantic)
   - Embedding dimension consistency

2. Test vector search accuracy:
   ```typescript
   // See templates/vector-search-tests.ts
   test('semantic search returns relevant results', async () => {
     const queryEmbedding = await generateEmbedding('machine learning');

     const { data } = await supabase.rpc('match_documents', {
       query_embedding: queryEmbedding
       match_threshold: 0.7
       match_count: 5
     });

     expect(data.length).toBeGreaterThan(0);
     expect(data[0].similarity).toBeGreaterThan(0.7);
   });
   ```

3. Test embedding operations:
   ```bash
   # Validate embedding pipeline
   npm test -- embedding-workflow.test.ts
   ```

4. Performance benchmarking:
   ```bash
   # Run performance tests
   bash scripts/benchmark-vector-search.sh [TABLE_NAME] [VECTOR_DIM]
   ```

### Phase 5: Realtime Features Testing

Test realtime subscriptions and presence:

1. Run realtime E2E tests:
   ```bash
   bash scripts/test-realtime-workflow.sh
   ```

   This validates:
   - Database change subscriptions
   - Broadcast messaging
   - Presence tracking
   - Connection handling
   - Reconnection logic
   - Subscription cleanup

2. Test database change subscriptions:
   ```typescript
   // See templates/realtime-tests.ts
   test('receives real-time updates on insert', async () => {
     const updates = [];

     const subscription = supabase
       .channel('test-channel')
       .on('postgres_changes'
         { event: 'INSERT', schema: 'public', table: 'messages' }
         (payload) => updates.push(payload)
       )
       .subscribe();

     await supabase.from('messages').insert({ content: 'test' });

     await waitFor(() => expect(updates).toHaveLength(1));
   });
   ```

3. Test presence and broadcast:
   ```bash
   # Run presence tests
   npm test -- presence.test.ts
   ```

### Phase 6: Complete Integration Tests

Run full workflow tests simulating real user scenarios:

1. Execute complete E2E test suite:
   ```bash
   bash scripts/run-e2e-tests.sh
   ```

   This runs:
   - User signup → profile creation → data CRUD → logout
   - Document upload → embedding generation → semantic search
   - Chat message → realtime delivery → read receipts
   - Multi-user collaboration scenarios
   - Error handling and recovery

2. Run test scenarios in parallel:
   ```bash
   # Run all test suites
   npm test -- --maxWorkers=4

   # Run specific workflow
   npm test -- workflows/document-rag.test.ts
   ```

3. Generate test reports:
   ```bash
   # Generate coverage report
   npm test -- --coverage

   # Generate HTML report
   npm test -- --coverage --coverageReporters=html
   ```

### Phase 7: CI/CD Integration

Setup automated testing in CI/CD pipelines:

1. Use GitHub Actions template:
   ```bash
   # Copy CI config to your repo
   cp templates/ci-config.yml .github/workflows/supabase-tests.yml
   ```

2. Configure test secrets:
   - Add `SUPABASE_TEST_URL` to GitHub secrets
   - Add `SUPABASE_TEST_ANON_KEY` to GitHub secrets
   - Add `SUPABASE_TEST_SERVICE_ROLE_KEY` for admin tests

3. Run tests on pull requests:
   - Automatic test execution on PR creation
   - Blocking PRs if tests fail
   - Test result reporting in PR comments

### Phase 8: Cleanup and Teardown

Clean up test resources after testing:

1. Run cleanup script:
   ```bash
   bash scripts/cleanup-test-resources.sh
   ```

   This removes:
   - Test users and sessions
   - Test data from tables
   - Temporary test databases
   - Test file uploads

2. Reset test database:
   ```bash
   # Reset to clean state
   supabase db reset --linked

   # Or use migration-based reset
   bash scripts/reset-test-db.sh
   ```

## Test Coverage Areas

### Database Testing
- **Schema Validation**: Table structure, columns, constraints
- **Migration Testing**: Up/down migrations, rollback safety
- **RLS Policies**: Access control, policy enforcement
- **Functions & Triggers**: Database logic, event handlers
- **Performance**: Query optimization, index usage

### Authentication Testing
- **User Flows**: Signup, login, logout, password reset
- **Provider Integration**: OAuth, magic links, phone auth
- **Session Management**: Token refresh, expiration, persistence
- **MFA**: Setup, verification, recovery codes
- **Security**: Rate limiting, brute force protection

### AI Features Testing
- **Vector Operations**: Insert, update, delete embeddings
- **Similarity Search**: Accuracy, relevance, threshold tuning
- **Index Performance**: HNSW vs IVFFlat, query speed
- **Hybrid Search**: Combined keyword and semantic search
- **Dimension Validation**: Embedding size consistency

### Realtime Testing
- **Subscriptions**: Database changes, broadcasts, presence
- **Connection Management**: Connect, disconnect, reconnect
- **Message Delivery**: Ordering, reliability, deduplication
- **Performance**: Latency, throughput, concurrent users
- **Cleanup**: Subscription disposal, memory leaks

### Integration Testing
- **Multi-Component**: Auth + D